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1.0  INTRODUCTION 


Robotic  vehicle  research  and  development  is  progressing  in  several  application  areas  which 
include  airborne,  space,  underwater,  and  terrestrial .  Terrestrial  applications  include  uses 
within  constrained  environments,  such  as  inside  buildings,  as  well  as  outdoor  applications, 
such  as  on  roads  or  across  open  areas.  Research  and  development  in  these  areas  is 
supported  by  industry  (oil  exploration,  warehousing,  etc.),  civilian  government  (DOE, 
NASA)  and  DOD  (Army,  Navy  and  Air  Force).  The  purpose  of  this  study  is  to  understand 
national  research  trends  within  certain  technical  areas,  specifically:  sensors,  perception 
algorithms,  cognitive  algorithms,  and  the  necessary  supporting  hardware  that  can  enable 
the  fielding  of  practical  autonomous  systems  for  land  warfare.  This  report  presents  the 
findings  of  this  study  regarding  the  status  of  national  research  trends  in  these  areas  and 
offers  some  suggestions  regarding  future  development  of  unmanned  vehicles  based  on 
these  trends. 

Technology  advances  in  microelectronics,  composite  materials,  electro-optics,  sensors, 
micromechanics,  computer  power,  and  artificial  intelligence  have  made  unmanned  military 
ground  vehicle  development  and  utilization  possible.  Although  there  is  a  vast  amount  of 
robotics  research  and  development  occurring,  there  has  never  been  an  attempt  to  organize 
an  understanding  of  current  research  trends.  Since  it  is  necessary  to  make  informed 
decisions  regarding  technologies  specifically  required  for  unmanned  ground  vehicles 
(UGVs),  it  is  important  to  understand  which  technologies  are  and  are  not  being  addressed 
by  other  application  areas  or  by  other  organizations. 

The  hazardous  environment  of  many  military  unmanned  vehicle  applications  necessitate 
some  very  different  requirements  than  those  of  civilian  applications.  In  many  cases,  a 
military  vehicle  will  have  to  operate  in  a  hostile  environment  on  paths  and  around  obstacles 
that  have  never  been  encountered  previously.  A  military  vehicle  must  also  be  prepared  to 
counter  or  avoid  detection,  threats  or  hostile  action,  where  an  intelligent  adversary  is 
purposely  attempting  to  thwart  the  vehicles  progress.  However,  Army/Marine  ground 
applications  also  have  similar  needs  in  technology  as  other  applications.  These  common 
technology  areas  include  real  time  data  links  for  communication  and  control,  image 
transmission,  and  a  capability  for  performing  some  level  of  autonomous  monitoring  and 
task  performance. 

Therefore,  the  objective  of  this  study  is  to  develop  an  understanding  of  national  research 
and  development  trends  in  robotics  and  how  these  trends  compare  to  needed  technologies 
for  the  unmanned  ground  vehicle  application.  Recommendations  will  also  be  made 
regarding  which  technologies  require  more  attention  than  they  are  now  being  given  in  order 
to  achieve  an  unmanned  vehicle  capability  for  the  ground  forces  and  assure  that  all 
technologies  required  are  matured  as  needed.  These  recommendations  will  account  for 
common  areas  of  research  where  duplication  need  not  occur,  such  as  communication  and 
image  transmission. 

Cost  and  mission  performance  are  the  predominant  considerations  in  the  development, 
implementation,  and  operation  of  unmanned  ground  vehicles.  In  almost  any  day  to  day 
operation,  manpower  is  the  most  costly  expense.  Cost  effective  UGVs  used  as  a  force 
multiplier  could  greatly  decrease  operational  costs.  Cost  and  other  systems  related  issues 
will  be  a  significant  factor  in  our  assessment  of  technologies  and  achievable  capabilities. 
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2.0  ANALYSIS  METHODOLOGY  FOR  UNMANNED 
GROUND  VEHICLE  OPERATION 

This  section  provides  a  framework  for  understanding  how  robotic  and  UGV  technologies 
are  to  be  classified  and  understood  in  the  context  of  this  study.  This  study  requires  that  an 
assessment  be  made  of  existing  technologies  in  order  to  develop  and  understand  desired 
research  and  development  goals.  Therefore,  an  exposition  of  how  robotics  is  viewed  is 
necessary  before  the  given  assessment  of  UGV  related  research  can  be  appreciated.  This 
exposition  will  be  presented  in  section  2.2. 

The  focus  of  the  present  study  is  on  how  to  achieve  autonomous  vehicle  navigation.  There 
is  some  debate  within  the  community  over  precise  definitions,  therefore  this  report  will 
reference  the  definitions  that  are  given  in  the  subsections  that  follow. 

What  will  be  presented  next  is  a  short  discussion  on  the  following  topics: 


•  Levels  of  Autonomy. 

•  Technologies  required  to  achieve  various  levels  of  autonomy. 

•  Assessment  methodology  and  factors  considered  in  the  assessment. 


2. 1  LEVELS  OF  AUTONOMY 

The  level  of  human  control  varies  between  two  extremes  that  we  call  direct  control  and 
indirect  control.  In  the  candidate  definitions  outlined  below,  control  implies  what  we  icnrs 
direct  control:  control  exerted  by  an  operator  on  the  actions  of  an  unmanned  ground  vehicle 
via  a  real-time  direct  link. 

The  definitions  must  be  further  clarified  by  stating  that  autonomy  is  not  a  single  concept 
For  purposes  of  this  study,  we  will  separate  rote  autonomy  from  intelligent  autonomy.  The 
former  refers  to  the  ability  to  carry  out  a  task  without  external  control,  but  only  m  a 
preprogrammed,  non-adaptive,  fashion.  Intelligent  (or  cognitive)  autonomy  implies  the 
ability  to  successfully  execute  tasks  in  real-world  and  therefore  uncertain  circumstances, 
such  that  the  task  environment  cannot  be  adequately  predicted  for  preprogramming.  For 
autonomous  navigation  applications,  we  are  concerned  with  intelligent  autonomy  and 
operating  in  an  uncontrolled  environment. 

In  the  lexicon  of  this  report,  there  are  six  forms  of  control  that  will  be  considered.  They  are: 

1)  Telepresence 

2)  Teleoperation 

3)  Teleassistance 

4)  Guided  robotics 

5)  Supervised  robotics 

6)  Autonomous  robotics 
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What  follows  is  a  more  descriptive  narrative  of  what  these  definitions  mean. 

•  Telepresence 

In  both  telepresence  and  teleoperation,  an  operator  has  full  control  of  all  vehicle  functions, 
from  navigation  and  driving  functions  through  mission  activities.  In  neither  case  does  the 
introduction  of  robotics  reduce  manpower  requirements:  both  are  limited  to  a  1:1 
operator:vehicle  ratio.  We  distinguish  the  two  modes,  not  by  the  degree  of  operator 
control,  but  by  the  sophistication  of  feedback  provided  to  the  operator. 

Telepresence  denotes  a  mode  of  operation  in  which  an  attempt  is  made  to  provide  full 
sensory  simulation  such  that  the  operator  feels  “present”  at  the  scene.  Telepresence  is 
therefore  characterized  by  elaborate  sensing  and  communications  requirements  in  order  to 
simulate  the  sensory  experience.  The  communications  load  and  sophistication  of  the 
operator/vehicle  interface  for  telepresence  suggest  that  this  type  of  operation  may  be 
reasonable  only  for  very  complex  or  delicate  tasks  to  be  conducted  in  hazardous 
environments,  i.e.  when  the  sensory  input  is  necessary  for  mission  execution  and  the  risks 
incurred  by  a  person  performing  the  mission  outweigh  the  cost  of  a  highly  sophisticated 
interface. 

•  Teleoperation 

In  teleoperation,  a  reduced  amount  of  feedback  is  provided  to  the  operator.  The  interface 
supplies  substantial  information  to  the  operator,  but  not  necessarily  in  the  form  which 
would  be  experienced  during  direct  operation.  For  example,  information  may  be  restricted 
to  a  single  modality  (e.g.,  video)  or  summarized  (e.g.,  a  pressure  readout  rather  than  tactile 
feedback).  Teleoperation  places  lower  demands  on  communication  channels  than 
telepresence  and  may  have  reduced  sensor  requirements.  However,  depending  upon  the 
information  needed  by  the  operator  teleoperation  may  require  more  on-board  computing 
for  preprocessing  sensor  and  vehicle  data. 

Examples  of  Teleoperation  are: 

-  TOV,  a  Marine  Corps  technology  demonstrator  of  the  late  80's  (see  section 
7.9.2). 

-  TMAP,  an  Army  technology  demonstrator  of  the  late  80’s  (see  section  7.8) 

-  The  Surrogate  Test  Vehicle  (STV),  used  in  the  pre-milestone  II  phase  of  the 
tactical  unmanned  ground  vehicle  (TUGV)  acquisition  program  to  assess,  jointly 
with  the  users  -  Army  infantry  and  Marine  Corps  -  operational  benefits  and 
liabilities  of  a  teleoperated  reconnaissance,  surveillance,  and  target  acquisition 
(RSTA)  UGV  and  to  serve  as  a  platform  for  demonstrating  in  a  military  system 
configuration  the  maturity  of  teleassistance  technologies  considered  for  the 
engineering  and  manufacturing  development  (EMD)  of  the  TUGV  (see  section 
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•  Teleassiston^  c 

In  teleassisted  operation,  the  vehicle  is  primarily  operator-controlled,  but  a  limited  set  of 
functions  are  left  to  the  control  of  the  machine.  In  principle,  these  functions  may  be 
mission-related  and/or  related  to  the  navigation  task.  All  actions  are  monitored  by  the 
operator  and  operator  intervention  is  possible  in  all  functions.  Because  of  the  level  of 
interaction  and  control  required  of  the  operator,  teleassistance  limits  an  operator  to  a  single 
UGV.  Relegation  of  some  tasks  to  machine  control,  however,  may  reduce  training 
requirements  and  operator  workload  and  may  improve  performance  in  stressful 
environments.  The  operator  interface  and  communication  requirements  can  be  relaxed  from 
those  of  teleoperation  with  a  corresponding  increase  in  on-board  computing  requirements. 


Examples  of  Teleassistance  are: 

-  JPL’s  Computer  Aided  Remote  Driving  (CARD)  programs,  either  CARD  I  or 
CARD  II.  The  vehicle  assists  the  operator  in  some  low  level  functions,  especially 
in  low  level  navigation  and  obstacle  detection.  Retrotraverse,  where  a  vehicle  can 
retrace  a  route  that  was  directed  by  a  human,  is  also  an  example  of  teleassistance. 
Retrotraverse  does  not  require  machine  intelligence  to  plan  the  original  route. 

-  Demo  I  (see  section  5. 1.2.1) 


•  Guided  Robotics 

Guided  robotics  is  distinguished  from  teleassistance  by  a  decrease  in  the  amount  of  control 
exerted  by  the  operator.  In  guided  robotics,  low  level  functions  are  entirely  machine- 
controlled  while  control  of  high  level  functions  is  retained  by  the  operator.  Certain 
functions  are  monitored  only  at  the  request  of  the  vehicle.  The  transition  from  teleassisted 
to  guided  robotics  incorporates  the  first  elements  of  true  autonomy.  The  vehicle  must  carry 
out  certain  functions  without  operator  oversight  and  therefore  is  required  to  monitor  its  own 
performance  well  enough  to  request  guidance  or  intervention  when  necessary.  In 
relieving  the  operator  of  low  level  functions,  guided  robotics  affords  the  first  opportunity 
for  force  multiplication  through  control  of  more  than  one  vehicle  by  a  single  operator. 

Examples  of  Guided  Robotics  are: 

-  JPL’s  Semi-autonomous  Navigation  (SAN)  program.  The  vehicle-landing  craft 
exploration  system  itself  takes  important  steps  in  path  planning  under  the 
guidance  of  a  human  remote  operator.  It  is  significant  that  this  vehicle  is  aimed  at 
space  exploration  in  which  cost  is  far  less  important  than  system  failure. 

-DARPA’s  Autonomous  Land  Vehicle  (ALV)  program 
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•  Super\’ised  Robotics 

Supervised  robotics  represents  a  substantial  increase  in  autonomy  over  guided  robotics. 

The  operator  retains  control  of  only  the  highest  level  functions,  which  may  be  exercised 
intermittently.  As  a  consequence,  control  of  multiple  vehicles  by  a  single  operator  becomes 
practical;  this  level  of  autonomy  has  the  potential  for  substantial  force  multiplication. 
Accompanying  the  increased  sophistication  of  the  machine  capability  is  a  significant 
reduction  in  communication  and  interface  requirements.  This  level  of  autonomy  enables  us 
to  partition  the  workload  between  the  machine  intelligence  (MI)  of  the  UGVs  and  the 
human  supervisor  such  that  the  capability  of  both  are  optimally  used;  e.g.,  all  routine  work 
is  done  by  MI.  The  human  handles  only  those  tasks  for  which  the  AI  technologies,  that  are 
required  to  approximate  human  capability  to  integrate  and  exercise  control  authority,  do  not 
yet  exist. 

Examples  of  Supervised  Robotics  are; 

-DOD  planned  DEMO  II  program  (see  section  5. 1.2.2) 


•  Autonomous  Robotics 

Autonomous  robotic  performance  is,  by  definition,  operation  without  external  control. 
Operational  communication  and  interface  requirements  are  not  at  issue,  except  from  the 
standpoint -of  testing  and  validation.  Autonomy  does  not  preclude  the  possibility  for 
preplanning  by  an  operator,  but  does  require  that  a  machine  be  able  to  carry  out  a  mission 
without  requiring  intervention.  The  requirements  for  intelligent  autonomous  capability  are 
not  completely  understood,  but  include  the  ability  to  adapt  to  environmental  changes  and  to 
acquire  missing  information.  The  requirements  for  rote  autonomy  are,  of  course, 
significantly  less. 

There  are  no  known  examples  of  Autonomous  Robotics  within  our  definitions. 


2.2  FUNCTIONAL  PROCESSES  FOR  AUTONOMOUS  OPERATION 

To  achieve  autonomous  movement,  there  are  three  functional  processes  required  that  are 
interrelated.  They  are: 

1)  Perception  :  Sensing  and  low  level  (high  volume)  data  processing  algorithms  and 
technologies. 

2)  Cognition:  High  level  (abstract)  symbolic  processing 

3)  Action:  Movement,  employment  of  reconnaissance,  targeting  or  weapons. 

Perception  includes  algorithms  and  technologies  that  allow  an  unmanned  vehicle  to  perceive 
(model)  its  surroundings,  its  position  and  identify  hostile  threats  and  other  situations  that 
occur  during  a  mission.  Cognition  is  a  classical  definition  that  includes  both  planning  and 
learning.  Cognition  allows  a  robot  to  understand  its  relationship  to  its  surroundings  and 
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how  to  plan  its  movements  in  order  to  achieve  its  objective.  It  could  be  said  that  this 
requires  awareness  and  judgement.  Planning  can  be  extended  to  also  mean  mission 
specific  understanding  besides  navigation,  for  instance,  understanding  when  to  engage 
weapons  or  employ  active  Reconnaissance,  Surveillance,  and  Target  Acquisition  (RSTA) 
sensors  to  best  advantage.  Learning  is  the  reasoning  required  for  automatic  planning  to 
achieve  specified  objectives.  Action  allows  a  robot  to  actually  achieve  its  goals,  such  as 
movement  over  rough  terrain  or  employment  of  sensors  and  weapons.  Research  related  to 
mobility  platforms  would  be  included  in  the  category  of  action  technologies. 


Figure  1  graphically  illustrates  the  relationship  between  these  three  pans  of  the  robotic 
process,  as  it  is  defined  in  this  repon. 


Figure  1.  Process  Required  for  Autonomous  Navigation. 

These  functional  processes  are  at  the  highest  level  of  abstractic  .  and  need  to  be  developed 
further  before  specific  research  strategies  can  be  analyzed. 


It  is  somewhat  intuitive  that  the  field  of  robotic  technology  for  ground  warfare  must 
necessarily  evolve  from  technologies  that  employ  high  levels  of  interaction  with  operators 
to  more  autonomous  applications.  Any  one  of  the  functional  processes  of  perception, 
cognition,  and  action  can  employ  human  intelligence,  especially  perception  and  cognition. 
Automating  these  functions  will  eventually  lead  to  the  “brilliant”  weapons  that  will  act  as 
effective  force  multipliers  for  the  ground  forces.  Figure  2  illustrates  the  relationship  of 
functional  processes,  human  interaction,  and  employment  of  a  ground  vehicle. 
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Figure  2.  Interaction  of  Human,  Machine,  and  a  Ground  Vehicle. 


Perception  involves  sensing  the  surroundings  of  a  UGV  and  processing  what  is  sensed  into 
a  form  that  facilitates  decision  making.  Technologies  that  will  contribute  to  this  process, 
then,  are  sensors,  sensor  preprocessing  of  data,  computer  vision  algorithms,  and  computer 
vision  processing  architectures.  To  understand  this  group  of  technologies,  some 
exposition  is  useful. 


A  common  assumption  in  the  robotics  community  is  the  utility  of  the  human  vision 
paradigm.  This  assumption  asserts  that  if  human  vision  can  be  mimicked,  then  the  obvious 
utility  of  our  superior  vision  system  can  be  brought  to  bear  in  robotic  applications.  The 
term  that  will  be  used  to  describe  this  pa  adigm  in  the  rest  of  this  report  is  computer  vision. 


As  a  means  for  understanding,  an  abstract  framework  for  computer  vision  can  be  described 
as  a  pyramid.  This  can  be  put  into  the  context  of  sensing  and  cognition  for  UGVs.  The 
basic  premise  is  that  there  are  levels  of  image  understanding,  beginning  with  basic 
processes  on  raw  data,  like  image  pixels,  up  to  the  most  abstract  reasoning,  such  as  the 
understanding  that  a  target  of  interest  is  present  in  an  observed  scene.  As  a  process 
progresses  in  stages  up  towards  understanding,  there  is  a  reduction  in  the  size  of  the  data 
set  required.  This  is  because  groups  of  data  are  abstracted  into  higher  levels  of 
understanding.  Figure  3  illustrates  this  perception  pyramid  notion. 
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Figure  3.  Perception  Pyramid  Showing  Progression  of  Low  Level  Data  to 
Higher  Level  Understanding  in  Stages. 

The  pyramidal  concept  of  perception  influences  ERIM’s  categorization  of  algorithms  and 
specialized  computing  architectures  for  image  understanding.  Quality  algorithms,  and 
computing  hardware,  are  required  at  all  levels  of  the  pyramid  in  order  to  adequately  address 
the  perception  needs  for  UGV  applications.  Specific  examples  of  existing  or  developing 
computer  vision  platforms  include  iWarp  (see  section  7.15),  ATCURE1,  ALADDIN1,  and 
IUA  (see  section  7.6).  These  architectures  provide  a  capability  to  increase  the  throughput 
at  each  of  these  levels  of  interaction.  For  instance,  iWarp  and  ATCURE  concentrate  on 
lower  level  pixel  processing  and  feature  extraction,  whereas  the  IUA  is  aimed  at  automating 
up  to  symbolic  processing. 

Human  interaction  is  also  possible  and  even  desirable  at  the  higher  levels  of  perception. 
Teleoperated  vehicles,  for  instance,  require  human  interaction  at  the  higher  levels  of 
“symbolic”  processing  in  order  to  perceive  obstacles  and  determine  desirable  paths.  The 
role  of  perception  in  the  decision  process  is  illustrated  in  Figure  4. 


1  ATCURE  and  ALADDIN  are  automatic  target  recognition  computing  systems  currently  under 
development  by  the  U.S.  Army. 


II 


Figure  4.  Role  of  Perception  in  Autonomous  Decision  Cycle 

Cognition,  another  focus  of  this  study,  can  also  be  understood  in  more  detail  in  terms  of 
two  major  elements: 

1)  World  and  environment  modeling. 

2)  An  intelligent  agent  that  operates  on  these  models  in  order  to  plan  and  direct 
action. 


It  is  understood  in  the  robotics  community  that  a  reasoning  machine  must  have  two  kinds 
of  models  on  which  to  operate:  World  models  and  environment  models. 


Many  versions  of  world  models  have  been  propounded.  They  are  described  in  terms  of 
map  coordinates,  analyzed  features,  and  prominent  landmarks.  World  models  are  typically 
known  a  priori  before  a  UGV  begins  a  mission.  Typical  scenarios  call  for  periodic 
updates  from  on-board  sensors  or  periodic  communication  with  an  outside  source. 
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Environment  models  are  different  than  world  models  both  in  their  horizons  and  in  time 
scale.  Environment  models  are  typically  not  known  a  priori,  or  are  only  sketchily  known. 
Environment  models  allow  for  local  path  planning  and  reactive  movement  to  locally 
encountered  threats  and  obstacles  (e.g.,  how  do  I  move  around  this  rock?  or  how  do  1 
avoid  this  enemy  armored  vehicle?).  Environment  models  include  simple  occupancy2  maps 
up  to  complex  CAD/CAM  descriptions  of  the  local  environment. 

Figure  5  illustrates  the  main  elements  of  cognition. 

Cognition  Module 


Intelligent  Agent 

-  Rules 

-  Procedures 

-  Learning  Mechanisms 


Models  as  data 


Where  am  1  in  the  world? 

World  Model 


Figure  5.  Elements  of  Cognition 

The  data  elements  for  the  models  required  in  the  cognition  module  are  provided,  not  by  the 
sensors,  but  by  the  perception  module  after  raw  sensor  data  have  been  processed. 
Naturally,  this  means  that  there  is  a  tight  coupling  of  methods  and  procedures  from  the 
perception  module  and  the  modeling  portion  of  the  cognition  module.  Figure  6  illustrates 


2Occupancy  maps  divide  the  world  into  simple  cubes  and  flag  whether  a  cube  is  partially  occupied  by  an 
object. 
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the  place  of  the  cognition  module  in  the  decision  cycle,  and  how  perception  feeds  the 
modeling  elements. 


Figure  6.  Coupling  of  Perception  and  Cognition  Through  Model  Elements 

The  intent  of  this  exposition  was  to  provide  a  top-level  understanding  of  the  entire  process 
of  autonomous  vehicle  navigation  and  employment.  Although  it  is  still  somewhat  abstract. 
Figure  6  provides  a  basis  for  understanding  technologies  that  could  solve  some  of  the 
needs  in  the  process.  For  instance,  sensing  for  perception  can  be  accomplished  by  either 
3D  active  ranging  or  3D  passive  stereo.  The  need  for  3D  sensing  is  dictated  by  the  need  in 
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the  environment  model  for  3D  information,  particularly  for  local  path  planning  and  driving 
execution,  thus  showing  the  linkage  that  dictates  requirements  on  the  sensor. 


2.3  TECHNOLOGIES  WITHIN  FUNCTIONALITIES  FOR  VARIOUS  LEVELS 
OF  CONTROL 

The  previous  section  has  expounded  on  the  entire  process  of  autonomous  control  from  an 
abstract  point  of  view.  The  purpose  of  this  section  is  to  identify  specific  functionalities  and 
technologies  that  could  potentially  perform  these  needed  functions.  This  will  then  provide 
an  infrastructure  of  understanding  the  importance  of  specific  technologies  are  to  the  UGV 
navigation  process. 

The  main  uses  of  military  autonomous  UGVs  will  be  in  support  of  hazardous  operations, 
including  providing  reconnaissance  and  surveillance  or  remote  target  acquisition.  Neither 
of  these  applications  will  emphasize  precise  or  delicate  movement  of  effector  units,  which 
is  a  major  area  of  development  in  industrial  robotics,  fire  control,  or  explosive  ordinance 
handling.  The  analysis  of  technologies  required  for  autonomous  UGVs  are  unique  in  their 
emphasis.  This  fact  is  reflected  in  the  following  discussion  regarding  functionalities  and 
technologies  required  to  achieve  autonomous  operation  in  ground  combat. 

A  careful  study  of  needed  technologies  for  various  levels  of  autonomy  has  led  the  authors 
to  derive  the  list  of  functionalities  presented  below.  The  definition  of  functionality  here 
means  that  the  needed  function  must  be  performed,  however,  in  many  cases,  there  exists 
more  than  one  technology  that  could  potentially  meet  that  functionality.  A  good  example  is 
3D  sensing:  either  passive  stereo,  active  laser,  MMW  radar  scanning,  or  an  array  of  simple 
sonar  ranging  devices  can  provide  three  dimensional  information  for  navigation. 

Listed  below  is  a  complete  list  of  functionalities  that  the  authors  have  derived  from  existing 
technology.  Functionalities  marked  with  an  *  are  those  that  are  the  focus  of  this  study. 

PeresptiQn..Eunctit>naiities 

•  Navigation  Sensors  * 

•  Sensor  Preprocessing  * 

•  Navigation  Perception  Algorithms* 

•  Computer  Vision  Architectures  * 

Cognition  Functionalities 

•  Navigation  Planning  * 

•  World  Model  * 

•  Environment  Model  * 

Action  Functionalities 

•  Communications  Link* 

•  Mission  Sensors 
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•  Mission  Related  Perception 

•  Effectors 

•  Man  Machine  Interface 

•  Vehicle  Model 

•  Integrating  Architectures 

•  Actuators 

•  Mobility  Platforms 

What  follows  are  some  definitions  and  examples  of  what  each  of  these  functionalities  mean 
for  those  which  were  analyzed  in  this  study. 

2.3.1  Definitions  of  Functionalities 

Navigation  Sensors 

The  actual  sensing  equipment  and  the  associated  data.  True  perception  is  a  combination  of 
sensors  and  algorithms,  which  is  reflected  in  these  two  categories.  Naturally,  there  are 
some  fuzzy  boundaries  between  these  definitions;  for  instance,  stereo  vision  will  be 
included  in  this  category  though  it  naturally  relies  heavily  on  intensive  computational 
procedure. 

Navigation  Perception  Algorithms 

Once  sensed  data  is  collected  and  processed,  the  data  must  be  converted  into  a  form  that  is 
usable  by  a  human  or  an  intelligent  planner.  Perception  algorithms  include  generic 
concepts  such  as  Depth  From  Focus  Blur  or  Depth  from  Optical  Flow3.  Perception,  then, 
is  the  ability  to  convert  sensor  data  into  a  model  for  use  by  an  intelligent  planner  (human  or 
machine). 

World  Model 

World  models  are  ways  of  describing  in  a  global  sense  where  the  vehicle  is.  Latitude  and 
longitude  is  one  way  of  describing  position,  position  relative  to  landmarks  would  be 
another. 

Environment  Model 

The  environment  model  provides  a  means  of  archiving  attributes  of  the  local  surroundings. 
Nearby  obstacles,  roads,  holes  and  hostile  threats  must  be  described  in  greater  detail  than  in 
the  world  model.  More  abstractly,  the  environment  model  accounts  for  the  state  of  the 
world  and  is  concerned  with  temporal  continuity.  This  allows  for  reactive  planning  to 
unexpected  events  and  threats. 


3  These  are  technologies  for  determining  range  from  image  characteristics  in  order  to  provide  3 -dimensional 
information  for  planning. 
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Navigation  Planning  Algorithms 

Navigation  planning  requires  maps  of  the  areas  to  be  traversed  for  use  in  planning 
algorithms.  Planning  algorithms  operate  on  the  data  provided  from  world  and  environment 
models.  These  algorithms  represent  intelligent  automata  and  as  such  are  replaced  by  human 
beings  at  lower  levels  of  autonomy  (telepresence,  teleassistance).  There  are  four  levels  of 
planning  defined  in  this  report:  mission  planning,  path  planning,  local  planning,  and 
obstacle  av.oidance/reaaive  planning. 

Mission  planning  requires  large  areas  of  the  terrain  to  be  surveyed  and  planned 
accordingly.  It  is  envisioned  that  100's  of  square  kilometers  will  be  included  by  a  mission 
planner,  but  at  coarser  resolution,  perhaps  100  meters  (DTED  Level  I).  At  this  level  the 
UGV  must  be  capable  of  communicating  to  command  and  control  and  other  elements  of  the 
force  structure. 

Path  planning  defines  the  optimal  path  for  a  UGV  to  take  to  accomplish  a  mission.  Path 
planning  requires  finer  detail  and  a  smaller  planning  horizon  than  mission  planning.  It  can 
be  envisioned  to  plan  over  an  area  on  the  order  of  10’s  of  square  kilometers  and  perhaps  on 
the  order  of  30  meter  resolution  (DTED  Level  II).  Path  planning  requires  communications 
from  the  local  planner  of  the  details  developed  therein  and  a  continual  updating  of  the  global 
path  plan. 

Local  planning  is  envisioned  to  have  a  planning  horizon  within  the  limit  of  mission  sensors 
(whether  RSTA,NBC,  or  targeting).  Local  planning  will  guide  the  UGV  to  the  best 
position  within  a  target  area  defined  by  the  path  planning  module.  It  can  be  envisioned  that 
resolution  requirements  might  be  on  the  order  of  30  cm  or  better,  with  a  horizon  within  the 
limits  of  the  mission  sensors.  This  map  for  local  planning  is  generated  at  the  UGV  by 
superimposing  of  local  geographic  information  provided  by  onboard  sensors  on  geographic 
map  files. 

Obstacle  avoidance/reactive  planning  has  an  even  shorter  planning  horizon,  within  the 
space  the  vehicle  has  to  avoid  or  react.  It  is  also  much  more  heavily  time  constrained,  and 
required  to  react  quickly  to  sensory  data  provided  to  it. 

Sensor  Preprocessing 

This  includes  camera  calibration,  warping,  coordinate  registration  and  other  mundane  but 
crucial  tasks  in  preparing  sensor  data  for  conversion  into  world  and  environment  models. 

Computer  Vision  Architectures 

This  functionality  refers  to  the  ability  to  process  both  low  level  (i.e.,  pixel)  and  high  level 
(i.e.,  symbolic)  data  in  a  time  frame  which  meets  the  UGVs  operational  requirements. 
Examples  of  specialized  architectures  include  Image  Understanding  Architecture  or 
Advanced  Target  Cueing  and  Recognition  Engine  (ATCURE).  Other  architectures  that 
could  be  considered  include  connection  machine  and  other,  more  general,  architectures. 

Communications 

The  planned  utilization  of  UGVs  as  force  multiplication  tools  will  require  communication 
between  multiple  UGVs  and  the  supervisory  command  and  control  network  in  which  they 
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arc  embedded.  An  optimum  communication  medium  must  allow  non-Iine-of-sight  (NLOS) 
communication,  must  be  secure,  and  must  be  able  to  support  variable  degrees  of 
maneuvering  ability  in  both  the  UGV  and  the  operator  control  unit 

Additionally,  a  need  for  real-time  video  communication  exists  primarily  at  the  lower  levels 
of  autonomy,  namely,  telepresence,  teleoperation,  and  teleassistance.  Under  these  types  of 
control,  a  human  operator  requires  video  feedback  from  the  UGV  to  effectively  close  the 
control  loop. 

2.3.2  Autonomy  Versus  Functionality 

As  with  any  analysis,  there  are  areas  of  ill-defined  functionality,  where  it  is  uncertain  what 
degree  of  functionality  is  required  for  a  certain  level  of  autonomy.  For  instance,  object 
recognition  and  even  obstacle  avoidance  could  be  useful,  even  necessary,  for  a  teleoperated 
vehicle.  However,  the  degree  of  such  functionality  would  not  be  as  severe  as  for  a  fully 
autonomous  vehicle. 

The  shaded  area  in  Figure  7  corresponds  to  the  functionalities  which  were  studied  as  part  of 
this  report  Question  marks  (?)  indicate  that  some  degree  of  a  particular  functionality  may 
be  required,  whereas  closed  circles  (•)  indicate  a  definite  requirement  for  the  level  of 
functionality  indicated. 
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Figure  7.  Levels  of  Autonomy  Versus  Required  Functionalities 

As  stated  before,  specific  technologies  already  exist  or  need  to  be  developed  in  order  to 
satisfy  the  functionality  requirements  depicted  in  Figure  7.  A  listing  of  known  technology 
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solutions,  either  demonstrated  or  conceptual,  will  aid  in  further  analyzing  the  current  state 
of  the  art  in  unmanned  ground  vehicles. 

For  ease  of  understanding,  technologies  have  been  grouped  hierarchically  where 
appropriate.  These  technologies  involve  both  hardware  and  software,  including  sensors, 
processors,  mobility  technologies,  planning  algorithms,  communications  and  man-machine 
interface  issues.  Not  all  technologies  listed  are  required  for  each  level  of  autonomy.  For 
instance,  an  elaborate  man-machine  interface  is  not  required  for  fully  autonomous 
operation. 

Following  is  a  listing  of  technologies  that  satisfy,  or  have  in  the  past  satisfied,  the  required 
functionalities  identified  above. 

•  Navigation  Sensors  (see  section  3.2) 

Passiyg 

-  FLIR 

-  Low  Light  Level  TV 

-  CCD  Video 

-  Stereo  (employing  any  one  of  many  passive  devices) 

-  Inertial  Guidance 
Active 

-  3D  active  systems  (laser  operating  in  the  IR  for  night,  or  MMW  for  all 

weather  and  night  regime) 

•  -  Array  of  Single  Point  Sensors  (such  as  small  sonars  or  small  IR  emitting 
devices  for  determining  range  within  a  narrow  FOV). 

Cwpcrati-vc 

-  Global  Positioning  System  (GPS) 

•  Navigation  Perception  (see  section  3.3) 

-  Shape  from  shading 

-  Perspective  transform 

-  Optical  flow 

-  Depth  from  focus  blur 

-  Depth  map  from  stereo 

-  Obstacle/Hole  detection 

•  Navigation  Planning  (see  section  3.4) 

Mission  Planning 

-  Neural  Net 

-  Potential/Gradient  Field 

.  -  Landmark  Based  Navigation 
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-  Case  Based  Reasoning 
Path  Planning 

-  Neural  Net 

-  Potential/Gradient  Field 

-  Landmark  Based  Navigation 

-  Case  Based  Reasoning 
Local  Planning 

-  Neural  Net 

-  Potential/Gradient  Field 

'  -  Landmark  Based  Navigation 

-  Case  Based  Reasoning 

Reactive  Planning/Obstacle  Avoidance 

-  Neural  net 

-  Gradient  Field 

-  Case  Based  Reasoning 

•  World  Model  (see  section  3.5) 

-  Cartographic 

-  Terrain  Models 

-  Landmark  Model 

•  Environment  Model  (see  section  3.6) 

-  2D  Occupancy  Maps 

-  3D  Occupancy  Maps 

-  Hierarchical  Occupancy  maps 

•  Sensor  Preprocessing  (see  section  3.7) 

-  Image  Registration 

-  Image  Warping 

-  Calibration 

•  Computer  Vision  Architectures  (see  section  3.8) 

•  Communication  Link  (see  section  3.9) 

-  Fiber  Optic  Cable 

-  High  Bandwidth  RF  Link 

-  SINCGARS  (w/  or  w/o  data  compression) 

•  Mission  Sensors 
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-  FLIR 

-  Low  Light  Level  TV 
*  Passive  Acoustic 

-  Seismic 

-  Electronic  Support  Measures 

-  Laser  Radar 

-  Radar 

-Millimeter  Wave  (active) 

•  Mission  Related  Perception 

-  Automatic  Target  Recognition 

-  Automatic  Target  Cueing 

•  Man  Machine  Interface 

•  Vehicle  Model 

•  Integrating  Architectures 

-  NIST  Robotic  Control  Structure 

-  NOSC  Modular  Robotic  Architecture 

-  CMU  Blackboard 

•  Actuators 

-  Vehicle 

-  Simple  Effector  (2  or  3  DOF) 

-  Manipulator  Arms  (6  DOF) 

•  Mobility  Platforms 

-  Wheeled 
-Legged 

-  Tracked 

2.4  ASSESSMENT  METHODOLOGY 

Unlike  many  other  robotics  applications,  the  forecasted  requirements  that  wi!!  be  placed  on 
unmanned  ground  vehicles  are  much  different  than  is  expected  for  an  industrial  robot 
UGVs  will  have  to  negotiate  difficult  terrain  that  is  only  partially  understood  beforehand, 
and  operate  in  an  environment  that  is  neither  stable  nor  controllable.  Not  least  of  the 
uncontrollable  aspects  of  a  UGVs  environment  is  the  presence  of  enemies  attempting  to 
detect  and  neutralize  it.  Therefore  coven  operation  must  be  a  prime  requirement.  AU 
technology  assessments  have  regarded  the  need  for  coven  operation  as  a  prime  operational 
consideration. 
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It  is  well  understood  that  fielding  a  practical  and  usable  system  requires  consideration  of 
many  issues  outside  purely  technical  concerns.  A  study  such  as  this  would  not  be  complete 
without  consideration  of  factors  that  relate  to  these  issues.  Technologies  that  will 
proficiently  solve  a  given  functionality  must  also  be  judged  on  other  factors  as  well,  such 
as  those  listed  below: 

•  Logistics 

•  Training 

•  Maintenance 

•  Life  Cycle  Cost 

•  Mobility 

•  Deployment 

•  User  Interface 

•  Survivability 

•  Effectiveness 

In  order  to  assess  the  technical  methods  that  could  solve  each  functionality,  we’ve  taken 
into  account  these  factors  listed  above.  However,  since  an  assessment  is  being  attempted  at 
a  point  in  time  where  the  technology  is  very  immature,  an  in-depth  study  of  these  system 
issues  is  premature.  It  is  nevertheless  appropriate  to  attempt  to  make  comparisons  and 
assessments  of  even  immature  technologies,  since  the  choice  of  a  technically  simple  and 
supportable  technique  over  a  complex  technique  will  influence  the  ease  with  which 
inexpensive  and  supportable  systems  will  eventually  be  developed. 

Ideal  requirements  on  components  of  a  robotic  system  entail: 

•  low  technical  risk  (for  yet-to-be-developed  technologies) 

•  high  proficiency  in  solving  the  problem 

•  low  complexity  for  ease  of  maintenance,  training,  and  survivability 

•  low  weight  for  design  flexibility 

•  low  cost,  both  acquisition  and  life-cycle 

Technical  assessment  factors  include: 

•  Technical  Risk. 

•  Proficiency. 

•  Projected  logistics  requirements. 

•  Personnel  requirements. 

•  Projected  cost. 


Consideration  of  these  technical  assessment  factors  was  fundamental  to  the  development  of 
the  discussion  in  the  next  chapter. 
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3.0  DEVELOPMENT  TRENDS  OF  NEEDED 
FUNCTIONALITY  AND  TECHNOLOGIES 


This  section  is  intended  to  condense  information  about  expected  maturation  of  each 
functionality  for  each  level  of  autonomy.  This  condensation  is  intended  to  provide  a  top 
level,  “quick  look”  view  of  functionalities  and  technologies.  Supporting  information  for 
each  matrix  element  is  provided  in  the  text  that  follows. 


3 . 1  OVERVIEW  OF  STATE  OF  THE  ART 

In  order  to  quickly  understand  the  state  of  the  art  compiled  in  this  report.  Figure  7  was 
revised  with  dates  showing  when  levels  of  functionality  will  achieve  sufficient  maturity  for 
demonstration  and  validation  (DEMVAL).  This  means  that  a  certain  technology  that  could 
fulfill  a  functionality  may  not  be  commercial-off-the-shelf  (COTS),  but  is  mature  enough  to 
be  incorporated  into  the  DEMVAL  plans  of  a  system  acquisition. 

It  must  be  understood  that  such  predictions  are  necessarily  difficult  and  contain  many 
uncertainties.  This  fact  is  well  understood  by  the  authors.  The  purpose  of  developing  such 
predictions  is  more  to  understand  how  functionalities  compare  with  one  another  more  than 
in  developing  absolute  predictions  of  when  functionalities  and  technologies  are  actually 
going  to  be  available. 

Figure  8  illustrates  the  assessments  for  technical  maturity  within  each  functional  area  that 
was  a  focus  of  this  study.  The  text  following  Figure  8  discusses  in  detail  each  box  and 
which  technologies  seem  most  promising  in  satisfying  the  given  functionality. 
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•  Reasonable  date 
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satisfied  given  cunrent  trends. 
Further  improvements  most 
likely  will  be  required. 


Figure  8.  Overview  Assessment  Functionalities  Required  for  All  Levels  of 

Autonomy 
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What  follows  is  more  in-depth  description,  definitions  and  justification  for  each  assessment 
in  die  matrix  in  Figure  8.  Each  functionality  will  be  discussed  individually  in  terms  of  its 
maturity  versus  each  level  of  autonomy. 

Each  assessment  is  keyed  by  functionality,  with  a  subheading  for  the  level  of  autonomy  so 
that  each  and  every  assessment  in  Figure  8  will  have  an  in  depth  discussion  regarding  how 
the  assessments  were  derived. 

3.2  NAVIGATION  SENSING 

Sensors  that  adequately  provide  information  for  the  perception  and  cognition  modules  are  a 
critical  component  in  a  UGV  system.  Advancements  in  electronic  sensing,  along  with 
advances  in  computer  hardware  and  software  techniques,  have  made  vision  possible  for 
UGV  applications.  Sensors  must  provide  data  that  allow  for  timely  construction/update  of 
a  sufficiently  accurate  3D  image  for  determining  the  local  navigation  path. 

Selection  of  navigation  sensors  must  consider  many  factors,  including: 

•  Navigation  perception  algorithms. 

•  Military  operational  requirements. 

•  Available  processors. 

•  Affordability. 

To  date,  sensing  modalities  that  have  been  most  common  in  development  UGV  systems 
are: 

•  3D  laser  scanners. 

•  Charge  couple  device  sensors  that  provide  either  black-and-white  or  color  two 
dimensional  images  (electro-optical  or  EO  sensors). 

•  The  same  sensors  in  multiple  arrangements  (pairs,  triples, ...)  to  provide  passive 
stereo  imagery. 

Each  of  these  sensing  modalities  have  their  drawbacks.  Active  scanners  like  3D  laser 
scanners  violate  a  basic  principle  of  warfare  in  that  they  may  give  away  the  position  of  a 
unit  by  alerting  an  enemy  of  its  presence. 

2D  electro-optical  sensors  are  limited  in  the  amount  of  information  they  can  provide,  even 
with  the  extension  to  color  sensors. 

Passive  stereo  may  be  limited  in  its  ability  to  provide  accurate  enough  information  at  a 
speed  that  is  consistent  with  movement  in  a  land  warfare  scenario.  Stereo,  in  a  research 
setting,  can  currently  provide  1  ffame/sec.  It  is  projected  that  this  can  be  improved  to  2-4 
frames/sec  in  the  near  future.  The  fundamental  limitation  to  stereo  is  correlation  versus 
accuracy.  Points  in  the  scene  must  be  correlated  in  each  stereo  pair.  The  more  distance 
between  cameras,  the  more  difficult  the  correlation  process.  However,  the  more  distance 
between  cameras,  the  more  accurate  the  depth  field  is  calculated.  This  fundamental 
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dichotomy  leads  one  to  project  there  may  be  a  fundamental  limitation  in  stereo  capability. 

To  date,  stereo  provides  at  least  an  order  of  magnitude  less  accuracy  than  active  laser 
scanning. 

A  suggestion  to  incorporate  a  multi-tiered  sensing  system,  employing  sensors  when  they 
are  most  appropriate  is  one  approach  to  solving  these  limitations.  For  instance,  stereo 
could  provide  gross  information  that  could  key  employment  of  active  scanners  at  selected 
times  and  places.  However,  multiple  elements  in  any  system  increases  the  likelihood  of 
failure  if  any  one  element  fails.  The  attendant  increased  system  complexity  and  expense 
must  be  carefully  considered  if  such  a  system  is  to  be  employed  in  harm’s  way. 

For  these  reasons,  we  do  not  believe  that  current  realizations  of  sensing  modalities  are 
adequate  for  the  higher  levels  of  autonomy,  and  that  there  is  some  question  that  the 
modalities  themselves  will  ever  be  fully  adequate  for  navigation  sensing  for  supervised  or 
autonomous  vehicles.  New  modalities  should  be  explored. 

Idspjsaen.es 

By  the  definition  given  for  telepresence,  a  high  level  of  human  feedback  is  required  for  the 
operator  to  have  the  impression  that  they  are  at  the  remote  location  of  the  vehicle.  In  its 
purest  sense,  this  requires  full  three  dimensional  vision,  smell,  touch  and  hearing. 

Although  research  in  this  area  is  ongoing  in  the  form  of  various  "virtual  reality”  programs 
in  universities,  it  will  be  many  years  before  practical,  low-cost  systems  can  be  fielded  for 
operational  use.  The  justification  for  this  is  that  only  highly  structured,  specialized 
applications  have  allowed  the  use  of  any  form  of  telepresence. 

However,  the  narrow  definition  we  have  developed  here  is  that  we  need  sensing  only  for 
navigation,  not  necessarily  for  mission  accomplishment  One  of  the  few  application  areas 
where  telepresence  might  be  useful  is  explosive  ordinance  disposal  (HOD)  and  perhaps 
mine  field  clearing,  again  in  the  mission  area  rather  than  navigation.  However,  DoD 
currently  has  plans  that  provide  for  this  mission  capability  without  resort  to  more  expensive 
telepresence  systems. 

Low  cost  and  reliable  three  dimensional  viewing  for  navigation  is  currently  available  and 
was  demonstrated  in  the  TOV  program  among  others.  3D  sound  is  being  investigated  and 
implemented  by  the  Armstrong  Laboratory  of  the  U.  S.  Air  Force,  and  will  be  ready  within 
a  few  years  for  implementation  in  an  UGV  setting.  Therefore,  the  assessment  is  that 
navigation  sensing  for  telepresence  in  the  sense  of  3D  stereo  vision  is  currently  available, 
3D  sound  will  soon  be  available.  Also,  INS  and  GPS  is  useful  and  is  also  available 
currently  for  global  understanding  of  position  for  navigation.  DEMO  I  will  provide  more 
information  on  the  need  for  stereo  versus  flat  video  for  teleoperated/telepresence  vehicles. 

Tglwpsratiofl 

Teleoperation,  in  the  navigation  area,  has  a  reduced  need  for  sensory  data  in  contrast  to 
telepresence,  and  by  the  definitions  of  this  report,  flat  video  would  be  sufficient.  TMAP 
and  the  current  envisioned  TUGV  design  will  employ  flat  video  for  navigation  purposes. 
Field  testing  and  user  comments  will  determine  the  need  for  transitioning  to  stereo  video  for 
navigation.  Day/night  capabilities  are  also  available  options  in  terms  of  either  FLIR  or  Low 
Light  Level  TV  (LLLTV).  FLER  offers  better  all  around  performance,  however  is  more 
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expensive  than  LLLTV.  Also,  INS  and  GPS  is  useful  and  is  also  available  currently  for 
global  understanding  of  position. 

Teleassistance 

By  the  definition  in  this  report,  what  is  required  in  new  sensing  capability  over 
teleoperation  is  the  need  to  detect  obstacles  and  impending  collisions  independent  of  the 
operator.  At  short  ranges,  this  is  possible  with  acoustic  and  point  IR  active  sensors.  JPL’s 
CARD  2  is  an  example  of  a  current  system  with  obstacle  detection  capabilities  inherent  in 
the  system,  though  CARD  2  accomplishes  this  through  automatic  data  extraction  of  the 
output  from  the  stereo  vision  sensor  rather  than  with  an  enhanced  sensor  suite.  Systems 
which  automatically  adjust  driving  parameters  based  on  the  path  to  be  driven  immediately 
ahead  of  the  vehicle  are  prime  examples  of  teleassisted  navigation. 

The  assessment  is  that  for  navigation  sensing,  this  capability  now  exists  for  teleassistance. 

Guided  Robotics 

Guided  robotics  requires  that  sensors  provide  data  that  enable  the  cognition  module  to 
recognize  obstacles  and  that  the  machine  be  capable  of  its  own  local  path  planning. 

Sensors  are  available  for  detecting  obstacles  that  then  require  operator  intervention.  Holes, 
ditches  and  other  depressions  still  pose  a  significant  problem  that  does  not  seem  to  be 
addressed  in  current  research  trends. 

Within  this  report's  definition,  road  following  is  a  constrained  version  of  guided  robotics. 
Speed  and  resolution  still  seems  to  be  a  problem  for  use  in  machine  intelligent  planners. 
Road  following  is  becoming  available  from  the  research  community,  as  demonstrated  by 
Carnegie  Mellon.  Off  road  navigation,  or  on  road  navigation  under  battlefield  conditions 
has  yet  to  be  demonstrated  in  the  research  community. 

To  date,  the  most  successful  sensor  for  use  in  any  form  of  cognitive  navigation  is  the  3D 
laser  sensor  (pioneered  by  ERIM).  The  other  type  of  sensor  used  is  the  range  finder  stereo 
system,  which  is  different  than  the  stereo  system  for  use  with  a  human.  The  range  finding 
stereo  provides  estimates  of  range  for  each  pixel  in  the  scene.  Current  state  of  the  art  is  a 
system  built  by  Matthies  at  JPL  which  provides  a  range  map  of  64  x  60  pixels  at  8-bit 
resolution  for  each  pixel. 

Passive  stereo  imaging  is  fundamentally  less  accurate  than  3D  laser  scanners.  Road 
following  systems  can  rely  on  flat  imagery  that  can  be  segmented  for  an  understanding  of 
where  a  road  is  and  where  the  vehicle  should  drive  itself. 

The  laser  scanner,  though  more  efficient  than  the  stereo  solution,  does  require  that  an  active 
signal  be  propagated.  In  the  hostile  environments  envisioned  for  frontline  UGVs  engaged 
in  RSTA,  the  question  of  covertness  and  active  scanners  is  a  serious  one. 

Supervised  and  Autonomous  Robotics 

Supervised  robotics  takes  the  operator  even  further  out  of  the  loop,  to  the  point  where  the 
machine  itself  is  making  most  decisions  related  to  navigation  and  movement,  allowing  user 
control  of  multiple  units.  This  requires  a  sensor  capable  of  assessing  traversability. 

Sensors  must  be  able  to  make  assessments  of  solidness  of  ground,  detect  true  obstacles 
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from  false  alarms,  as  well  as  detect  holes  and  other  depressions.  To  date,  these  have 
remained  difficult  problems  to  solve. 

The  two  most  popular  modalities  for  more  autonomous  navigation,  stereo  vision  and  3D 
laser  scanning,  both  have  inherent  limitations  that  may  not  be  overcome  by  further 
engineering  refinements.  Stereo  vision  may  always  inherently  lack  speed  and/or  resolution 
for  adequate  perception.  3D  laser  scanning  will  always  have  the  potential  of  compromising 
coven  operations  required  for  primary  operations. 

The  assessment  for  supervised  sensing  is  the  year  2000,  for  autonomous  2005.  These 
assessments  are  based  on  current  research  trends. 

Summary  of  Findings  and  Recommendations 

For  reasons  detailed  above,  we  do  not  believe  that  current  realizations  of  sensing  modalities 
are  adequate  for  the  higher  levels  or  autonomy,  and  that  there  is  some  question  that  the 
modalities  themselves  will  ever  be  fully  adequate  for  navigation  sensing  for  supervised  or 
autonomous  vehicles.  New  modalities  should  be  explored. 

As  will  be  stated  again  in  section  6.0,  the  search  for  new  modalities  or  the  refinement  of 
existing  modalities  needs  to  be  driven  by  an  overall  systems  approach  that  seems  to  be 
lacking  in  current  UGV  research  and  development  Each  development  community  (sensor, 
planning,  communications,...)  seems  to  be  deriving  requirements  independent  of  each 
other.  This  seems  to  be  less  true  in  sensors,  but  is  still  a  significant  factor. 

Specific  examples  can  be  pointed  to  where  an  integrated  approach  is  lacking.  For  instance, 
traversability  assessments  are  crucial  for  off-road  navigation,  even  when  the  human  is  in 
direct  control  of  a  vehicle  from  a  remote  location.  An  operator  or  automated  driving 
algorithm  will  need  to  know  if  the  patch  of  ground  ahead  of  the  unit  is  swampy  or  soft  or 
vegetation  covering  an  impassable  ditch  or  barrier.  Current  EO  and  active  sensors  cannot 
provide  this  kind  of  information,  nor  have  they  attempted  to.  Furthermore,  resolution,  area 
coverage  and  frame  rate  requirements  have  not  been  algorithm  driven. 

Sensing  modalities  that  could  be  explored  include  novel  passive  stereo,  active  and  passive 
polarization  exploitation,  active  and  passive  milli-meter  wave,  multi-color/multi-wavelength 
band  sensing  as  well  as  more  system-exotic  ideas  like  multi-sensor  fusion  for  synergistic 
exploitation  of  observables. 

To  date,  requirements  on  the  sensor  have  not  been  driven  by  what  is  required  by  the 
cognition  module.  The  issue  of  requirements  on  the  sensor  suite  driven  by  an  analysis  of 
what  is  required  by  cognition  module  has  not  been  fully  addressed. 

3 . 3  NAVIGATION  PERCEPTION 

As  it  has  been  defined  (in  2.3.1),  perception  does  not  apply  to  the  first  three  categories 
(telepresence,  teleoperation,  teleassistance),  since  the  human  operator  is  largely  providing 


28 


S^ERIM 


the  needed  interpretation  of  sensor  data4.  This  is  why,  as  will  be  discussed,  three 
dimensional  stereo  is  so  effective  when  presented  to  a  human  operator  and  is  less  so  when 
a  machine  algorithm  is  required  to  compute  3D  stereo  range  maps. 

The  basic  problem  of  perception  is  the  fact  that  it  is  a  non-invertable  problem.  A  non- 
invertable  problem  in  this  instance  means  that  many  conditions  can  create  any  one  image. 
Said  another  way,  the  world  can  be  inferred  many  ways  from  a  single  image.  What  is 
inferred  from  an  image  to  be  a  hole  may  be  a  mud  puddle,  a  rock  or  a  shiny  piece  of 
asphalt,  each  of  which  could  create  exactly  the  same  kind  of  image.  Ways  of  dealing  with 
non-invertable  problems  in  general  are  difficult.  A  complete  solution  to  perception  at  the 
highest  levels  of  autonomy  will  require  an  understanding  of  how  to  treat  the  non-invertable 
class  of  problem. 

The  current  leading  candidates  of  algorithms  for  perception  algorithms  are: 

•  Neural  net 

•  Potential/Gradient  Field  methods 
« Landmark  based  navigation 

Other  candidates  that  have  longer  realization  horizons  include: 

•  Case  based  reasoning 

Neural  nets  work  by  attempting  to  simulate  the  workings  of  the  inner  brain.  Neural  nets 
are  trained  on  a  set  of  data  (images,  or  3D  images)  and  taught  what  to  avoid  and  perceive  in 
the  gaining  set.  An  advantage  of  neural  nets  is  their  apparent  generality.  A  disadvantage  is 
the  inability  to  trace  out  WHY  a  particular  decision  was  made  (neural  nets  have  no  recovery 
ability).  Another  disadvantage  is  the  need  to  have  a  large  training  set  available  in  order  to 
train  the  algorithm. 

Potential/Gradient  field  systems  develop  a  map- based  model  that  takes  into  account  factors 
that  are  important  to  the  unit,  such  as  traversability,  iines-of-sight,  vegetation  or  cover  and 
other  factors.  To  date,  this  has  proven  to  be  the  most  useful  algorithm  for  local  path 
planning. 

Landmark  based  navigation  uses  local  landmarks  to  allow  for  fuzzy  navigation  through 
terrain  that  is  only  partially  understood.  The  definition  of  a  landmark  can  change  from 
scenario  to  scenario.  This  algorithm,  admittedly,  competes  with  other  kinds  of  technology, 
such  as  GPS  and  does  not  provide  mission  dependent  information  like  Gradient  Field  does. 

Case  based  reasoning  is  a  new  algorithmic  approach  that  takes  a  lawyer  like  approach  to 
reasoning.  A  series  of  scenarios,  or  cases,  are  stored  into  a  memory  unit  and  accessed  by  a 
software  agent  when  a  new  situation  arises.  Reasoning  about  what  to  do  next  is  based  on 


4We  assume  that  obstacle  avoidance  in  teleassisted  is  by  acoustic  or  point  sensor  means,  therefore  the  basic 
capability  is  in  the  sensor  rather  than  in  the  interpretation  of  sensed  data. 
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the  historical  experience  from  the  stored  cases.  Case  based  reasoning  will  not  be  available 
for  practical  systems  before  the  turn  of  the  century. 

Higher  quality  sensing  systems,  that  provide  dense,  quality  information  (thus  reducing  the 
non-invertability  of  perception)  will  alleviate  the  difficulty  in  developing  perception 
algorithms.  This  necessarily  entails  that  sensing  modalities  and  perception  techniques 
should  be  developed  hand  in  hand. 

Computer  vision  is  the  primary  means  by  which  navigation  perception  will  be  achieved. 
Computer  vision  is  still  a  growing  discipline.  A  general  model  of  computer  vision  has  been 
generally  accepted,  in  which  higher  levels  of  abstraction  reduce  large  amounts  of  raw  pixel 
data  into  smaller  amounts  of  generalized  knowledge  (refer  to  figure  3).  Many  techniques 
have  been  devised  and  researched  for  autonomous  vehicle  navigation,  but  robustness  is  still 
lacking  in  the  sense  that  any  one  technique,  or  even  any  combination  of  techniques,  can 
solve  all  perception  problems  required  for  unaided  cross  country  navigation.  Active  3D 
sensing  has  proven  to  be  the  best  overall  sensing  modality,  primarily  because  it  reduces  the 
burden  on  the  perception  algorithms  required  to  convert  the  raw  sensor  data  into  something 
that  a  cognitive  module  can  find  useful.  This  is  because  it  provides  more  information  than 
passive  systems  to  date,  thus  reducing  the  non-invertability  of  the  perception  task. 

Guided  Robotics 

Perception  for  guided  robotics  has  been  partially  solved,  as  demonstrated  by  Camegie- 
Mellon’s  road  following  successes.  Transition  into  the  more  general  case  is  required,  and 
may  be  achievable  within  the  next  few  years,  especially  as  a  result  of  the  DEMO  D  related 
efforts  as  well  as  the  Mars  Rover  efforts  at  JPL. 

One  issue  to  be  considered  in  this  assessment  of  “solved”  perception  is  the  fact  that  active 
laser  sensing  has  enabled  the  success  demonstrated.  As  mentioned  earlier,  active  sensing 
will  not  be  completely  satisfactory  for  most  primary  missions  envisioned  for  UGVs. 

Supervised  Robotics 

Perception- for  the  supervised  category  will  be  at  least  partially  satisfied  given  a  successful 
DEMO  D,  therefore  the  date  1996  is  assessed.  It  is  not  expected  that  a  full  and  satisfactory 
rating  of  success  can  be  stated,  even  given  a  successful  DEMO  n,  since  many  issues  will 
undoubtedly  remain.  However,  by  1996,  enough  enabling  technologies  should  be 
demonstrated  that  supervised  autonomous  vehicles  should  at  least  be  partially  realized. 

Autonomous  Robotics 

Perception  algorithms  and  computing  hardware  for  vehicles  that  can  freely  roam 
unstructured  environments  will  not  be  ready  in  the  foreseeable  future.  The  requirement  for 
error  recovery  and  rapidly  responding  to  quick,  unforeseen  events  makes  this  not  as  likely 
in  the  near  future.  Again,  this  is  in  part  because  of  sensors  that  cannot  provide  enough 
information  fast  enough  regarding  range,  traversability,  and  other  features  of  the 
environment  to  reduce  the  non-invertability  of  the  basic  perception  problem. 

The  assessment,  given  current  research  trends  in  sensors,  is  that  perception  for  fully 
autonomous  navigation  is  most  likely  not  ready  until  the  year  2010.  Current  research 
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trends  in  sensors  is  improved  versions  of  stereo  and  improved  versions  of  3D  laser 
scanners. 

Summary  of  Findings  and  Recommendations 
For  the  higher  three  categories  of  autonomy,  general  comments  apply.  These  are: 

•  Road  following  has  been  demonstrated  in  a  research  setting.  Within  this  report, 
this  is  considered  a  limited  subset  of  guided  robotics.  Given  the  success 
demonstrated,  road  following  could  be  implemented  in  an  advanced  system 
development  within  a  short  time. 

•  Obstacle  detection  is  more  developed  than  detecting  holes  or  depressions  (even  for 
teleoperation). 

•  Fast  enough  depth  map  for  limited  applications  from  stereo  camera  pairs  should  be 
available  by  the  year  1994,  but  accuracy  and  resolution  will  be  limited,  requiring 
powerful  perception  algorithms  to  understand  the  output  from  such  sensor  data. 
Fast  enough  is  on  the  order  of  2-4  hertz.  Resolution  accuracy  is  on  the  order  of 
1-5  inches  of  spatial  resolution  in  order  to  detect  obstacles. 

It  is  the  author's  contention  that  sensing  modalities  that  can  provide  consistent  and  complete 
information  are  a  major  reason  for  current  limitations  in  robotic  functionality.  Current 
perception  algorithms  are  overtaxed  and  slow,  not  because  of  fundamental  limitations  in 
and  of  themselves,  but  because  there  is  not  rich  enough  information  being  received  from 
current  sensing  modalities.  If  this  information  were  available,  programs  exploiting  higher 
levels  of  autonomy  could  conceivably  be  entering  DEM/VAL  by  1996. 

Again,  like  all  areas  of  robotics,  an  integrated  approach  to  defining  requirements  and 
developing  systems  should  greatly  affect  the  development  of  perception  algorithms.  A 
greater  appreciation  of  sensing  modalities  and  their  potential  information  content  would 
positively  affect  the  development  of  these  algorithms. 

3.4  NAVIGATION  PLANNING 

Recall  the  definitions  for  the  four  levels  of  planning: 

Mission  planning  requires  large  areas  of  the  terrain  to  be  surveyed  and 
planned  accordingly.  It  is  envisioned  that  100’s  of  square  kilometers  will  be 
included  by  a  mission  planner,  but  at  coarser  spatial  resolution,  perhaps  100 
meters  (DTED  Level  I). 

Path  planning  defines  the  optimal  path  for  a  UGV  to  take  to  accomplish  a 
mission,  path  planning  requires  finer  detail  and  a  smaller  planning  horizon.  It 
can  be  envisioned  to  plan  over  an  area  on  the  order  of  10’s  of  square 
kilometers  and  perhaps  on  the  order  of  30  meter  resolution  (DTED  Level  II). 

Local  planning  is  envisioned  to  have  a  planning  horizon  within  the  limit  of 
mission  sensors  (whether  RSTA,NBC,  or  targeting).  Local  planning  will 
guide  the  UGV  to  the  best  position  within  a  target  area  defined  by  the  path 
planning  module.  It  can  be  envisioned  that  resolution  requirements  might  be 
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on  the  order  of  30cm  or  better,  with  a  horizon  within  the  limits  of  the  mission 
sensors. 

Obstacle  avoidance! reactive  planning  has  an  even  shorter  planning  horizon, 
within  the  space  the  vehicle  has  to  avoid  or  react.  It  is  also  much  more 
heavily  time  constrained,  required  to  plan  quickly  to  sensory  data  provided  to 
it. 

Planning  applies  only  to  supervised,  guided  and  autonomous,  since  it  is  again  assumed  that 
humans  will  provide  planning  abilities  at  the  lower  levels  of  autonomy.  TTiis  functionality 
has  been  addressed  by  artificial  intelligence,  neural  net  and  other  disciplines  or 
combinations  of  these.  The  specific  techniques  that  were  addressed  in  this  report  are: 

-  Neural  Net 

-  Potential/Gradient  Field 

-  Landmark  Based  Navigation 

-  Case  Based  Reasoning 

To  date,  there  has  been  little  actual  demonstration  of  any  of  these  techniques  towards 
planning  the  route  of  an  unmanned  vehicle  to  traverse  unstructured  territory.  Gradient  field 
techniques  appear  to  hold  the  most  promise.  More  work  has  been  done  on  route 
planning/obstacle  avoidance  than  on  higher  level  mission  planning  for  robotics,  hence  its 
relative  maturity  versus  other  functionalities.  An  important  point  to  consider,  however,  is 
that  there  is  a  large  body  of  mission  planning  and  rehearsal  for  manned  and  active  units  that 
could  be  applied  to  this  problem.  This  has  been  recognized  in  the  organization  and 
planning  for  Demo  II. 

The  potential/gradient  field  method  offers  the  best  near  term  solution  for  all  levels  of 
planning.  Gradient  field  approaches  develop  a  vector  map  of  potentials  or  gradients  that 
advises  the  planner  how  to  achieve  its  objective,  no  matter  where  it  is  on  the  ground  (as 
opposed  to  specific  route  planning  that  requires  a  planner  to  execute  a  specified  route  on  the 
ground).  Gradient  fields  can  be  constructed  using  all  information  required  to  guide  an 
autonomous  vehicle,  including  placement  of  roads,  ground  cover,  elevation,  slope, 
traversability,  field-of-view  of  opposing  enemy  units  and  other  factors.  More  importantly, 
new  information  updated  continuously  from  on-board  sensors  and  incoming 
communications  can  be  used  to  make  real-time  decisions.  David  Payton  at  Hughes 
Research  Lab,  as  well  as  Borenstein  and  Koren  and  Jean-Claude  LaTombe  have  studied 
this  field  extensively. 

Landmark  based  navigation  (as  well  as  qualitative  navigation)  also  shows  promise.  This 
technique  uses  distinctive  landmarks  or  “perceptual  events”  in  order  to  determine 
approximate  or  relative  position.  However,  although  useful,  position  determination 
actually  competes  with  GPS  or  INS  and  may  not  provide  the  kind  of  value  decision  making 
the  gradient  field  methods  provide.  The  University  of  Massachusetts,  University  of 
Michigan  and  ADS  have  pursued  research  in  this  field.  Another  drawback  is  that 
landmarks  are  heavily  context  dependent,  i.e.  a  landmark  in  Saudi  Arabia  would  not  be  a 
landmark  in  Central  Europe. 
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Guided  Robotics 

Local  Planning 

To  date,  the  gradient  field  methods  offer  the  best  near  term  solution  to  local 
planning.  Neural  net  technology  may  prove  superior,  but  is  still  a  new  and 
evolving  field.  Local  planning,  being  similar  in  horizon  to  reactive  planning,  has 
evolved  further  than  more  abstract  levels  of  path  and  mission  planning.  Case  based 
reasoning  may  prove  superior,  but  is  a  field  in  its  infancy  and  cannot  be  considered 
for  the  short  term.  It  is  expected  that  the  gradient  field  work  developed  for  DEMO 
II  will  be  useful  in  this  category  (if  demonstrated  successfully).  Therefore,  a 
reasonable  date  is  1994-1996,  given  current  research  trends.  This  is  especially  true 
in  tjiis  category  since  some  level  of  human  intervention  is  still  expected. 

An  example  of  successful  local  planning  is  the  road  following  capabilities 
demonstrated  by  CMU’s  NAVLAB  n. 

Reactive  Planning/Obstacle  Avoidance 

The  assessment  is  that  potential  field  methods  are  at  a  higher  level  of  development 
than  neural  nets  and  that  fieldable  examples  can  be  produced  by  1993.  Obstacle 
avoidance  was  demonstrated  in  the  ALV  program  and  the  CMU  NAVLAB  currently 
performs  obstacle  avoidance.  Although  improvement  can  be  accomplished, 
perhaps  through  neural  nets,  it  can  be  stated  that  a  partial  capability  now  exists. 

Supervised  Robotics 

Mission  Planning 

Mission  planning  in  this  category  will  have  less  severe  requirements  because  of 
more  human  interaction  supervising  the  progress  of  the  vehicle.  An  intelligent 
planner  is  expected  to  be  more  of  an  aid  to  a  human  operator  who  will  make  all  final 
decisions.  In  this  case,  the  potential  field  techniques  will  be  very  useful  in 
providing  advice.  In  contrast  to  neural  nets,  potential  field  techniques  can  describe 
why  a  decision  was  made.  Neural  nets  may  not  be  available  for  mission  planning 
until  1995  and  no  convincing  approaches  have  been  demonstrated  for  mission 
planning.  Landmark  based  navigation  has  been  assessed  not  to  be  available  until 
1998  although  landmark  recognition  and  assisted  navigation  techniques  are 
expected  to  be  available  by  1995.  Since  Hughes  Research  Labs  expect  to  have  a 
version  of  the  potential  gradient  method  available  for  DEMO  II,  the  date  in  this 
category  is  1996. 

Path  Planning 

Path  planning  is  expected  to  be  more  automated  than  mission  planning  at  this  level 
of  automation,  yet  the  planning  horizon  and  resolution  will  be  similar  to  the  mission 
planning  horizon.  The  tradeoff  is  a  less  difficult  problem  in  terms  of  goal 
achievement  versus  more  processing  required.  A  reasonable  date  for  successful 
automatic  path  planning  is  expected  to  lag  human  assisted  mission  planning.  1996 
is  the  assessed  date  for  this  category. 
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Local  Planning 

To  date,  the  gradient  field  methods  offer  the  best  near  term  solution  to  local 
planning  for  this  level  of  autonomy.  Neural  net  technology  may  prove  superior, 
but  is  still  a  new  and  evolving  field.  Local  planning,  being  similar  in  planning 
horizon  to  reactive  planning,  has  evolved  further  than  more  abstract  levels  of 
planning.  Case  based  reasoning  may  prove  superior,  but  is  a  field  in  its  infancy 
and  cannot  be  considered  for  the  short  term.  It  is  expected  that  the  gradient  field 
work  developed  for  DEMO  II  will  be  useful  in  this  category  (if  demonstrated 
successfully).  Therefore,  a  reasonable  date  is  1994-1996,  given  current  research 
trends.  This  is  especially  true  in  this  category  since  some  level  of  human 
intervention  is  still  expected. 

Again,  as  in  guided  robotics,  road  following  is  an  example  of  successful  local 
planning. 

Reactive  Planning/Obstacle  Avoidance 

The  assessment  is  that  potential  field  methods  are  at  a  higher  level  of  development 
than  neural  nets  and  that  fieldable  examples  can  be  produced  by  1993.  Obstacle 
avoidance  was  demonstrated  in  the  ALV  program  and  the  CMU  NAVLAB  currently 
performs  obstacle  avoidance.  Although  improvement  can  be  accomplished, 
perhaps  through  neural  nets,  it  can  be  stated  that  a  partial  capability  now  exists. 

Autonomous  Robotics 

Mission  Planning 

The  higher  level  of  planning  and  intelligence  required  for  fully  autonomous  mission 
planning  makes  this  category  more  difficult  to  predict  It  is  expected  that  gradient 
fields  will  again  be  the  most  likely  candidate,  however  neural  nets  and  especially 
case  based  reasoning  may  be  the  most  efficient  techniques  at  this  level.  Given  such 
uncertainty,  the  best  prediction  is  year  2000  (or  longer  if  case  based  is  required). 

Path  Planning 

Comments  are  the  same  as  under  supervised. 

Local  Planning 

Comments  are  the  same  as  under  supervised,  except  the  added  requirements  for 
fully  autonomous  will  prolong  the  full  satisfaction  of  this  functionality  until  the  year 
2010. 

Reactive  Planning! Obstacle  Avoidance 

The  assessment  is  that  potential  field  methods  are  at  a  higher  level  of  development 
than  neural  net  and  that  fieldable  examples  can  be  produced  by  1993.  Obstacle 
avoidance  was  demonstrated  in  the  ALV  program  and  the  CMU  NAVLAB  currently 
performs  obstacle  avoidance.  Although  improvement  can  be  accomplished, 
perhaps  through  neural  nets,  it  can  be  stated  that  a  partial  capability  now  exists. 
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Improvement  required  for  fully  autonomous  navigation  may  not  be  available  before 
the  year  2010. 

Summary  of  Findings  and  Conclusions 

Mission  planning  for  UGV  applications  has  not  been  addressed  as  a  separate  field  of  study, 
however  it  may  not  need  to  be,  given  the  great  body  of  work  in  mission  planning  as  a 
whole.  Undoubtedly  subtleties  and  variations  on  the  main  thrust  of  mission  planning  will 
be  required  for  UGVs  but,  as  a  whole,  the  field  may  well  be  in  hand  for  the  lower  levels  of 
autonomy  up  through  supervised. 

Autonomous  mission  planning,  as  we  have  defined  it,  has  not  been  addressed.  It  is 
expected  that  some  variant  of  a  gradient  field  approach  will  prove  useful.  However,  it  will 
likely  be  the  year  2000  before  a  level  of  satisfaction  will  be  achieved. 

3.5  WORLD  MODEL 

World  modeling  is  closely  related  to  mission  and  path  planning,  therefore  it  applies  only  to 
the  highest  levels  of  autonomy.  The  most  relevant  kinds  of  technologies  that  apply  seem  to 
be  available  from  the  Defense  Mapping  Agency  (DMA)  in  formats  like  Digital  Terrain 
Elevation  Data  (DTED),  Digital  Feature  Analysis  Data  (DFAD)  or  other  products,  like 
Interim  Terrain  Data  (1TD).  Landmark  models,  such  as  used  in  Landmark  Recognition  and 
Qualitative  Navigation,  requires  context  sensitive  world  models  and  will  require  more 
development  before  they  are  available. 

The  need  for  world  modeling  beyond  these  kinds  of  representations  is  still  to  be 
determined.  Research  and  development  in  this  area  is  still  new,  however  DEMO  II  is  slated 
to  deploy  mission  planning  tools  that  will  allow  some  understanding  of  how  well  these 
kinds  of  representations  will  be  suited  for  an  automatic,  intelligent  planner. 

It  is  assessed  that,  if  successful,  a  suitable  world  model  can  be  developed  by  1996  based 
on  DMA  available  products. 

3 . 6  ENVIRONMENT  MODEL 

Environment  modeling  is  more  difficult  than  world  modeling,  because  the  cognition 
module  of  an  autonomous  vehicle  requires  vastly  more  fine  resolution  and  more  kinds  of 
data  in  order  to  detect  and  avoid  hazards,  threats  and  obstacles.  In  guided  robotics,  this 
requirement  is  not  severe,  becoming  more  severe  for  supervised  and  most  severe  for 
autonomous  robotics.  Existing  environment  models  include  2  and  3D  occupancy  maps 
(pixels  and  voxels5),  wireframe  models,  faceted  models,  complicated  CAD/CAM  models 
based  on  constructive  solid  geometry  techniques  and  more  symbolic  representations  of  the 
local  environment,  such  as  hierarchical  representations  based  on  voxel  maps. 


5  Voxels  arc  "Volume  Elements"  and  are  used  to  subdivide  the  world  into  uniform  cubes. 

*  i 
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CMU  has  used  simple  2D  occupancy  maps  for  obstacle  avoidance.  University  of  Michigan 
has  explored  3D  occupancy  maps  (voxels  and  octrees6)  as  well  as  CAD/CAM  models  and 
symbolic  environment  representations.  To  date,  the  lower  level  representations  of  the 
environment  (without  resort  to  symbolic  representations),  have  been  demonstrated. 

Again,  as  cannot  be  stressed  enough,  there  is  a  tight  coupling  of  the  environment  model, 
the  perception  algorithm  which  updates  the  environment  model  and  the  planning  agent  that 
uses  the  environment  model  to  guide  the  actions  of  the  vehicle.  In  the  case  of  gradient  field 
planning  methods,  discussed  above,  the  environment  model  must  include  estimates  of 
elevation,  slope,  traversability,  obstacles  and  any  other  considerations  in  the  environment 
that  must  be  taken  into  account  during  planning.  Sensors  must  be  deployed  that  can  sense 
these  qualities  of  the  environment,  and  a  perception  algorithm  exist  that  can  extract  these 
qualities  from  the  sensed  data. 

Since  there  is  such  a  tight  coupling  between  sensing,  perception,  and  planning; 
development  of  an  appropriate  model  cannot  precede  in  any  real  sense  the  development  of 
perception  algorithms  (it  can  for  a  world  model,  since  on-board  perception  is  not  as  highly 
needed  for  mission  planning,  which  requires  a  world  model). 

Guided  Robotics 

2D  occupancy  maps  have  proven  useful  in  simple  obstacle  avoidance  by  CMU.  3D 
occupancy  maps  axe  useful  in  avoiding  obstacles  and  mapping  holes  and  depressions. 

Since  2D  occupancy  maps  have  been  used  to  date,  the  assessment  is  that  this  functionality 
is  partially  met  Development  of  higher  forms  of  environmental  models  will  be  required. 
3D  voxels  or  octree  methods  will  be  available  by  1995  and  hierarchical  forms  will  be  ready 
by  1997-2000. 

Supervised  Robotics 

2D  occupancy  maps  have  p»CYf"  useful  in  simple  obstacle  avoidance  by  CMU.  3D 
occupancy  maps  are  useful  in  avoiding  obstacles  and  mapping  holes  and  depressions. 

Since  2D  occupancy  maps  have  been  used  to  date,  the  assessment  is  that  this  functionality 
is  partially  met  Development  of  higher  forms  of  environmental  models  will  be  required. 
3D  voxels  or  octree  methods  will  be  available  by  1995  and  hierarchical  forms  will  be  ready 
by  1997-2000. 

Autonomous  Robotics 

2D  occupancy  maps  have  proven  useful  in  simple  obstacle  avoidance  by  CMU.  3D 
occupancy  maps  are  useful  in  avoiding  obstacles  and  mapping  holes  and  depressions. 

Since  2D  occupancy  maps  have  been  used  to  date,  the  assessment  is  that  this  functionality 
is  partially  met.  Development  of  higher  forms  of  environmental  models  will  be  required. 


6Octrees  are  similar  to  voxels,  but  subdivide  the  world  into  non-uniform  cubes.  The  data  structure  used  to 
relate  the  cubes  is  an  octal  tree  structure,  hence  the  term  octree. 
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3D  voxels  or  octree  methods  will  be  available  by  1995  and  hierarchical  forms  will  be  ready 
by  1997-2000.  However,  because  of  the  extraordinary  demands  on  the  environment 
model,  refinement  of  existing  models  or  new  models  might  have  to  be  developed  before 
autonomous  navigation  can  be  achieved.  The  assessment  for  this  level  of  modeling  is  the 
year  2010. 

Summary  of  Findings  and  Recommendations 

Current  trends  in  environmental  modeling  will  solve  the  problems  of  the  higher  levels  of 
autonomy  in  the  projected  future.  Occupancy  maps  with  information  "tagged"  inside  cells 
appear  to  have  the  brightest  future.  The  greatest  challenge  will  be  in  the  fully  autonomous 
field,  since  so  much  information  must  be  encapsulated  and  be  accessed  in  short  periods  of 
time. 

3.7  SENSOR  PREPROCESSING 

Sensor  preprocessing,  as  defined  in  this  report,  is  the  processing  and  “massaging”  of  raw 
sensor  data.  Artifacts  due  to  lens  aberrations,  mis-alignment  and  mis-focus  that  can  be 
corrected  can  be  handled  by  sensor  preprocessing.  This  would  include  image  warping, 
electronic  artifacts  due  to  non-uniform  detectors  and  registration  as  well.  Sensor 
preprocessing,  then,  applies  equally  to  all  levels  of  autonomy. 

Sensor  preprocessing  can  be  a  significant  systems  engineering  issue  when  individual 
systems  are  designed  and  fabricated,  especially  systems  that  incorporate  new  sensing 
modalities.  However,  no  significant  research  must  be  accomplished.  Sensor 
preprocessing  from  a  theoretical  point  of  view  is  not  an  issue  that  will  hinder  UGV 
development  and  deployment,  as  significant  as  it  is  from  a  systems  point  of  view. 

3 . 8  COMPUTER  VISION  ARCHITECTURES 

Specialized  computer  vision  hardware  will  still  be  needed  in  the  foreseeable  future  to 
accomplish  the  higher  levels  of  autonomy,  not  withstanding  the  rapid  pace  with  which 
general  purpose  computing  is  increasing  in  power.  The  processor  must  be  able  to  partition 
its  workload  of  manipulation,  RSTA,  and  effector  chores  m  a  way  that  these  are  completed 
in  a  timely  manner.  There  is  a  question  of  whether  these  tasks  can  be  accomplished 
concurrently.  Since  the  needed  computational  capability  may  not  be  available  in  time,  it  is 
necessary  to  address  the  issue  of  assigning  priorities  to  situations  and  actions,  e.  g.  slow 
down  navigational  mobility  while  RSTA  is  conducted. 

Given  the  lack  of  robustness  in  available  computer  vision  techniques,  it  is  predicted  that  a 
combination  of  many  computer  vision  techniques,  each  tuned  to  a  particular  aspect  of  the 
problem,  will  be  required.  This,  along  with  the  sheer  volume  of  computation  required  in 
image  domains,  will  require  computing  power  more  than  that  available  from  general 
purpose  computers. 

Recognizing  this,  DARPA  has  funded  the  Image  Understanding  Architecture  (IUA),  a  joint 
effort  between  the  University  of  Massachusetts,  Amherst  and  Hughes  Research 
Laboratory.  This  project  is  scheduled  to  produce  a  working  prototype  by  1996. 
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Guided  Rotoacs 

Because  of  the  reduced  need  for  information  throughput,  especially  at  the  symbolic  level,  it 
can  be  stated  that  computer  vision  architectures  currently  exist  for  guided  robotics.  A  good 
example  of  this  assertion  is  the  use  of  iWarp  in  CMU’s  road  following  NAVLAB.  it  has 
been  stated  by  Jon  Webb  and  Jill  Crisman  of  CMU  that  the  iWarp  machine  is  sufficient  for 
autonomous  road  following. 

Extending  'guided  robotics  to  more  general  settings  will  probably  require  greater  computing 
power  than  iWarp,  especially  at  the  symbolic  and  knowledge-based  levels  of  abstraction. 
Hence,  the  qualification  of  1992-1996  and  the  half  shading  in  this  category  in  Figure  8 

Supervised  Robotics 

The  IUA  mentioned  above  is  being  developed  to  support  DEMO  D,  which  is  an  example  of 
supervised  robotics  within  the  definitions  of  this  report  Since  a  working  prototype  is 
scheduled  for  1996,  the  date  listed  in  this  box  reflects  the  projected  development  of  the 
IUA. 

Autonomous  Robotics 

Although  the  planning  and  cognition  modules  of  a  fully  autonomous  vehicle  may  require 
specialized  architectures  of  their  own,  it  is  speculated  by  the  authors  that  if  the  IUA  is 
proven  to  work  well  enough  for  DEMO  II,  it  will  also  prove  sufficient  for  the  computer 
vision  aspects  of  this  level  of  autonomous  robotics.  Hence,  the  date  is  the  same. 

3.9  COMMUNICATION 

Communication  is  a  fundamentally  important  UGV  component,  not  only  in  support  of 
navigation,  but  also  in  support  of  specific  mission  applications.  In  the  case  of  specific 
missions,  it  is  intuitive  that  to  be  completely  effective,  a  UGV  must  be  able  to  communicate 
the  intelligence  it  has  collected  and/or  the  degree  of  success  of  its  particular  mission.  The 
communication  requirements  associated  with  UGV  mission  operations  arc  perceived  (by 
the  authors)  to  be  less  constrained  than  those  associated  with  real-time  vehicle  navigation 
and  control. 

The  principle  driver  of  navigation-related  communication  requirements  rests  primarily  at  the 
lower  levels  of  autonomy  (i.e.,  telepresence,  teleoperation,  and  teleassistance)  At  these 
jevels,  a  human  operator  is  integrally  involved  in  vehicle  control.  The  amount  of  human 
interaction  necessary  to  operate  the  UGV  at  these  lower  levels  places  significant  burdens  on 
the  volume,  speed,  and  accuracy  capabilities  of  the  communication  system.  As  reflected  in 
Figure  8,  these  burdens  are  not  necessarily  outside  the  capabilities  of  existing 
communication  technologies.  The  difficulty,  or  challenge,  arises  from  the  need  to 
superimpose  military  operational  requirements  on  the  communication  equipment  Specific 
military  operational  requirements  include  the  following  : 

•  a  desire  to  maintain  a  low  detectability,  or  signature,  of  the  transmitter. 

•  the  robustness  of  the  system  against  electronic  counter-measures 

•  the  security  of  information  exchange 
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•  the  allocation  of  a  suitable  communication  band 

•  the  timeliness  of  the  transmission 

•  the  ability  to  transmit  and  receive  with  a  non-line-of-sight  (NLOS)  capability. 

There  are  many  options  to  be  traded  off  between  communication  system  capabilities, 
military  operational  requirements,  and  the  functional  requirements  of  the  UGVs  specific 
mission  and  operational  regime  (i.e.,  level  of  autonomy).  This  study  focused  specifically 
on  the  anticipated  communication  requirements  to  support  the  UGV  navigation  function. 
General  areas  of  communication  technology  are  summarized  with  comments  on  the  trade¬ 
offs  between  their  capabilities  an  i  some  of  the  military  operational  requirements  mentioned 
previously. 

The  following  paragraphs  briefly  state  the  anticipated  communication  requirements  for  each 
of  the  levels  of  UGV  autonomy  defined  in  this  report.  In  general,  the  burden  on  UGV 
communication  equipment,  in  terms  of  functionality  and  operating  within  military 
requirements,  is  inversely  related  to  the  levels  of  autonomy,  i.e.,  communication 
requirements  are  least  stringent  at  the  highest  levels  of  autonomy.  With  increasing  levels  of 
autonomy  there  is  a  reduction  in  sensor  feedback,  to  the  operator,  and  also  the  migration  of 
more  control  functions  to  the  vehicle.  Subsequently,  as  the  level  of  UGV  autonomy 
increases,  it  will  become  easier  to  specify  communication  equipment  which  meets  a  broader 
set  of  military  operational  requirements 

Telepresence 

Telepresence,  by  definition,  involves  a  very  high,  and  complex,  level  of  human-UGV 
interaction  in  order  to  simulate  operator  presence  at  the  remote  location.  Subsequently,  the 
communication  burden  imposed  by  this  level  of  human  involvement  is  itself  quite  high.  It  is 
reasonable  to  anticipate  multi-channel,  high  bandwidth  communication  systems  will  be 
necessary  to  accommodate  multiple  images  and  sensors  (tactile,  sound,  seismic,  etc),  as 
well  as  high  bandwidth  control  of  the  platform’s  mobility  and  effector  capabilities. 
Continuous  communication  is  required  for  UGVs  operating  in  this  regime. 

Two  existing  communication  technologies,  fiber  optics  and  wideband  radio  frequency 
(RF),  currently  represent  the  best  options  for  meeting  the  communication  load  of  UGVs 
operating  in  the  telepresence  regime.  However,  both  have  significant  limitations  on  the 
UGVs  maneuverability  (hence,  navigability).  Specifically,  fiber  optics  requires  a  physical 
tether  (essentially  an  umbilical)  and  wide  band  RF  systems  require  a  clear  line-of-sight  to 
exist  between  the  vehicle  and  the  control  station. 

Tdeapcratipn 

A  principle  difference  between  telepresence  and  teleoperation  is  the  reduction  of  sensory 
feedback  to  the  operator.  Hence,  the  quantity  of  transmitted  sensory  data  is  reduced  and 
there  is  also  reason  to  expect  that  a  lower  data  fidelity  may  be  tolerated.  The  principle 
requirement,  from  a  navigation  standpoint,  is  the  need  to  transmit  good  quality  and  timely 
video  to  support  operator  control  of  the  platform.  There  are  significant  tradeoffs  between 
the  update  rate  of  transmitted  image  frames  and  the  safe  speed  of  forward  progress  of  the 
UGV.  Since  the  UGV  is  still  entirely  operator  controlled,  it  is  anticipated  that  it  will  be 
necessary  to  transmit  multiple  scenes  to  support  all  levels  of  UGV  navigation  (i.e.,  obstacle 
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avoidance,  local  navigation,  and  path  planning).  Additionailj ,  by  definition, 
communication  between  the  UGV  and  the  operator  control  s.ation  must  be  continuous. 

As  mentioned  previously,  the  assessment  is  that  communication  technology  now  exists  to 
facilitate  teleoperation.  However,  significant  remaining  work  involves  evaluating  how 
tradeoffs  imposed  by  military  operational  requirements  effect  the  teleoperation  of  the  UGV. 
Of  particular  interest  is  the  tradeoff  between  data  compression  technology  and  the  ability  of 
the  operator  to  effectively  control  the  vehicle. 

Teleassistance 

This  level  continues  the  trend  in  reducing  sensing  requirements  and,  hence,  communication 
requirements,  required  for  UGV  navigation.  While  there  is  a  defined  distinction  from 
teleoperadon  requirements,  the  UGV  is  still  primarily  operator  controlled,  hence  the  need  to 
provide  imagery  for  operator-assisted  navigation  still  exists.  Since  the  vehicle  has  some 
low  level  navigation  capability,  a  single  scene  may  be  sufficient  for  operator  interaction.  As 
with  the  previous  two  levels,  communication  must,  necessarily,  be  continuous.  However, 
it  is  at  this  level  that  data  compression  technology  can  begin  to  make  a  significant  impact  on 
balancing  between  the  capabilities  of  existing  communication  equipment  and  military 
operational  requirements.  Since  the  vehicle  has  some  rudimentary  navigation  and  control 
capability,  the  video  frame  rate  might  be  reduced  without  a  corresponding  change  in  vehicle 
speed. 

Guided  Robotics 

At  this  level,  a  sufficient  amount  of  UGV  control  remains  with  the  platform  such  that 
compromises  between  existing  communication  technology  and  military  operational 
requirements  are  readily  reached.  Hence,  for  this  level  of  autonomy,  and  those  higher. 
Figure  8  indicates  a  date  of  1992,  with  no  shading.  A  key  difference  from  lower  levels  of 
autonomy  is  that  the  need  for  communication  might  become  intermittent,  with  no  resulting 
loss  in  vehicle  navigation  capability. 

Supervised  ancLAutonomous  Robotics 

The  communication  requirements  to  support  UGV  navigation  at  these  two  levels  are 
virtually  identical,  and  hence,  for  discussion  purposes  here,  have  been  combined.  The 
principle  requirement  of  continuous  communication,  which  existed  at  the  lower  levels,  is 
significantly  relaxed  at  the  supervised  and  autonomous  levels.  From  a  navigation 
standpoint,  both  vehicles  have  a  capability  to  operate  substantially  on  their  own,  hence 
when  communication  with  the  operator  station  becomes  desirable,  the  UGVs  have 
flexibility  for  choosing  the  location  (and  hence,  the  direction),  as  well  as  the  length  and 
timeliness  of  the  transmission. 

Technology 

The  shaded  communication  areas  shown  in  Figure  8,  for,  derive  from  the  need  for 
continuous  communication  between  the  UGV  and  the  operator  control  unit  (OCU).  This 
need  exists  at  the  lower  levels  of  vehicle  control,  specifically,  telepresence,  teleoperation, 
and  teleassistance.  Under  these  types  of  control,  a  human  operator  requires  good  quality 
and  timely  video  feedback  from  the  remote  vehicle  to  effectively  close  the  navigation 
control  loop.  As  the  control  and  navigation  capabilities  are  migrated  to  the  vehicle  (as 
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planned  for  in  the  higher  levels  of  UGV  autonomy),  the  need  for  continuous 
communication  becomes  more  a  function  of  the  vehicle’s  particular  mission. 

Three  existing  communication  technologies,  based  on  fiber  optic,  wide-band  radio 
frequency  (RF),  and  narrow-band  RF,  have  been  used  extensively  to  transmit  video. 
However,  each  of  these  technologies  has  significant  limitations  in  terms  of  supporting 
operator  navigation  and  control  of  UGVs.  Most  of  the  limitations  are  well  recognized  by 
the  UGV  community  and,  thus,  have  led  to  several  RDT&E  programs  designed  to  increase 
the  effectiveness  and  utility  of  each  technology  The  following  paragraphs  identify  the 
recognized  limitations  of  these  three  technologies,  provide  a  summary  of  activities  designed 
to  address  these  limitations,  and,  in  some  cases,  discuss  potential  options. 

Fiber  Optic  Communication  Technology 

Most  teleoperated  UGV  programs  have  used  fiber  optic  technology  as  the  primary  means 
for  communicating  with  the  vehicle.  Fiber  has  been  used  previously  in  the  Fiber  Optic 
Guided  Missile  (FOG-M)  and  the  Robotic  Obstacle  Breaching  Assault  Tank  (ROBAT).  The 
advantages  of  using  fiber  include  high  bandwidth,  secure(including  low  detectability),  and, 
to  a  lessor  extent,  beyond  line-of-sight  communications.  The  major  disadvantage  is  the 
existence  of  a  physical  tether  (essentially  an  umbilical)  between  the  OCU  and  the  UGV. 
This  tether  restricts  the  maneuvering  ability  of  the  UGV  through  certain  key  types  of 
terrain,  such  as  forests  and  rivers.  The  tether  also  places  restrictions  on  the  relocation  of  the 
OCU.  Operational  concerns,  such  as  vulnerability  of  the  tether  to  damage  (possibly  caused 
by  other  vehicles  driving  over  it)  or  entanglement,  as  well  as  the  complexity  associated  with 
payout  and  retrieval,  are  also  valid. 

Since  the  physical  tether  is  a  fundamental  requirement,  virtually  all  UGV  RDT&E  activity 
related  to  fiber  addresses  operational  concerns.  The  Naval  Ocean  Systems  Command 
(NOSC),  in  collaboration  with  CECOM,  are  noted  in  particular  due  to  their  work  to 
ruggedize  fiber  optic  cable,  as  well  as  improve  deployment  and  retrieval  mechanisms. 
Additionally,  some  private  sector  organizations  (e.g.,  Optelecom)  have  developed  and 
demonstrated  routine  (i.e.,  golfcart)  cable  deploy  and  retrieve  mechanisms.  On  other 
issues,  such  as  river  fording,  ideas  have  been  suggested,  however  there  is  a  recognized 
lack  of  test  data  to  substantiate  them. 

The  state  of  the  art  in  fiber  optics  has  reached  maturity,  as  far  as  the  UGV  application  is 
concerned.  In  the  near  term,  there  may  be  requirements  for  some  low  level  RDT&E 
investment,  associated  with  operational  concerns,  however,  large  new  investments  in  the 
technology  are  not  warranted  This  is  based  in  part  on  the  fact  that  other  research  trends  are 
toward  higher  levels  of  UGV  autonomy  which  do  not  necessarily  require  continuous 
communication  (from  the  navigation  and  control  standpoint).  Additionally,  operations  at 
longer  distances,  including  those  behind  enemy  lines,  will  best  be  served  by  alternate,  RF- 
based,  technologies. 

Wide-Bandwidth  RF  Communication  Technology 

Wide-bandwidth  RF  communication  has  generally  been  viewed  as  providing  an  important 
secondary  (fiber  being  the  primary)  communication  link  which  enables  the  teleoperator  to 
retrieve  the  platform  or,  potentially,  continue  the  mission.  Wide-band  RF  systems  are  free 
from  physical  tethers  and  have  been  used  to  transmit  good  quality,  real-time  video.  The 
principle  constraint  in  using  wide-band  RF  is  that  a  clear  line-of-sight  (LOS)  must  be 


41 


3>ERIM 


maintained  between  the  transmit  and  receive  antennas.  This  constraint  restricts  maneuvering 
the  vehicle  through  forests,  over  hills,  and  behind  buildings.  However,  system  concepts 
have  been  proposed  which  use  various  types  of  signal  relay  stations  (e.g..  Unmanned  Air 
Vehicles,  satellites,  or  even  other  UGVs)  to  help  alleviate  some  aspects  of  this  problem. 

Additionally,  some  SBIR  activities,  which  will  be  part  of  DEMO  I,  specifically  address 
simultaneous  transmission  of  multiple  video  signals  and  light  weight,  low  cost, 
autotracking  antennas. 

A  system  concept  study  looking  at  the  utility  of  a  satellite  constellation  as  signal  relay 
stations  was  performed  by  the  ARMY  Missile  Command  (MICOM).  Such  an  approach  is 
desirable  as  it  reduces  the  LOS  problem  and  supports  the  use  of  Super  High  Frequencies 
(SHF)  and  Extremely  High  Frequencies  (EHF).  These  spectral  regions  are  not  as  crowded 
as  lower  regions  and,  thus,  frequency  allocation  for  UGV  operations  may  be  easier  to 
obtain.  However,  no  Low  Earth  Orbit  (LEO)  constellation  presently  exists  which  could 
support  UGV  operation.  Some  commercial  constellations  are  planned  (e.g..  Iridium,  by 
Motorola,  for  a  world-wide  cellular  capability)  which  may  be  applicable,  however  it  is  not 
clear  that  these  would  provide  100%  communication  availability  in  all  potential  theaters  of 
interest. 

Narrow-Bandwidth  Communication  Technology 

Advances  in  machine  intelligence  technology  will  certainly  remove  the  need  for  transmitting 
real  time  video  for  operator  control  of  a  UGV.  However,  the  need  for  video  communication 
will  probably  remain  in  support  of  mission-related  functions.  As  with  wide  bandwidth 
communication,  an  optimum  narrow  bandwidth  medium  must  allow  non  line-of-sight 
communication,  must  be  secure,  and  must  be  able  to  support  various  degrees  of 
maneuvering  ability  in  both  the  remote  vehicle  and  the  operator  control  unit 

In  wide  bandwidth  RF  systems,  the  LOS  dependence  is  primarily  due  to  the  high  operating 
frequencies  of  these  systems.  The  operating  frequency  can  be  reduced  in  order  to  achieve 
better  non-LOS  performance,  however  the  available  bandwidth  required  to  transmit  live 
video  does  not  exist  due  to  the  extreme  congestion  of  the  frequency  spectrum. 

A  proposed  solution  involves  reducing  the  bandwidth  necessary  to  transmit  a  usable  video 
image,  through  the  use  of  video  compression  technology.  The  development  of  this 
technology  is  presently  receiving  a  great  deal  of  worldwide  attention  and  funding  because 
of  a  broad  applicability  in  areas  ranging  from  high-definition  TV  (HDTV)  through 
computer  video  applications  to  satellite  and  telephone-line  data  compression.  Video 
compression  RDT&E,  in  support  of  UGV  operations,  has  centered  on  two  major  areas, 
technique  and  human  factors. 

Several  organizations  are  presently  investigating  the  feasibility  of  controlling  UGVs  be 
means  of  low  data  rate,  narrow-band,  RF  links.  One  alternative  is  to  use  a  packet-switched 
radio,  such  as  the  single  channel,  ground/air  radio  system  (SINCGARS).  However, 
SINCGARS,  which  has  a  low  probability  of  intercept  and  a  low  probability  of  detection, 
can  only  support  transmission  of  16  kB/s.  While  this  rate  may  be  adequate  for  command, 
control,  and  status,  data  compression  techniques  are  required  to  support  video 
transmission.  Specifically,  in  order  for  a  system  such  as  SINCGARS  to  be  effective  in  this 
application,  the  video  signal  must  be  compressed,  transmitted,  and  then  decompressed.  In 
order  to  investigate  SINCGARS  as  a  viable  option  for  real-time  image  transmission,  DOD 
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has  on-going  research  efforts  directed  toward  the  development  of  robust  video  compression 
techniques. 

Another  particular  technology  involves  a  feedback  limited  control  system  (FEL1CS)  which 
uses  selectable  tradeoffs  between  the  level  of  video  compression  and  the  bandwidth  of  the 
video  channel.  Specifically,  the  video  may  be  transmitted  via  a  high-rate  wide-band  video 
channel  or  a  very  narrow-band  channel  that  restricts  the  video  frame-rate  to  well  under  one 
frame  per  second.  DEMO  I  will  provide  more  information  on  the  utility  of  FELICS 
technology. 


Additionally,  other  efforts  are  aimed  at  understanding  how  much  video  information  must  be 
transmitted  for  the  safe  operation  of  the  vehicle.  Groups,  such  as  Sandia  National 
Laboratories,  have  conducted  operator  tests  in  an  attempt  to  understand  some  of  these 
issues. 

Summary  of  Findings  and  Recommendations 

The  communication  technologies  required  for  military  application  of  UGVs  are,  for  the 
most  part,  currently  available,  hence  the  projected  date  of  1992  shown  for  this  functionality 
in  Figure  8.  The  challenges  arise  from  the  superposition  of  military  operational 
requirements  on  the  UGV  communication  equipment  These  challenges  are  particularly 
acute  at  the  lower  levels  of  UGV  autonomy  which  require  continuous  communication  for 
effective  operation  (from  a  navigation  standpoint).  To  reduce  the  burden  which  continuous 
communication  imposes  at  the  lower  UGV  levels,  the  main  thrust  of  near-term  research 
should  be  data  compression  technologies.  This  thrust  should  include  compression  efforts  in 
both  data  processing  and  transmission  techniques.  It  is  believed  that  this  technology  will 
continue  to  be  useful  as  the  UGV  platform  evolves  to  a  more  intelligent  mechanism.  The 
level  of  support  should  reflect  the  fact  that  this  area  already  enjoys  significant  worldwide 
interest  and  investment.  Hence,  we  recommend  that  continuing  efforts  in  support  of  UGV 
objectives  include  evaluation  of  new  hardware  and  software,  as  they  appear,  and  the 
RDT&E  of  component  technologies  which  are  specific  to  UGV  needs.  We  further 
recommend  that  the  organizations  involved  in  UGV  video  compression  RDT&E  be 
encouraged  to  communicate  and  identify  synergistic  opportunities. 

Additionally,  technology  that  supports  beyond  line-of-sight  operation,  without  a  physical 
tether,  needs  to  be  explored  further.  Examples  include  the  use  of  satellites,  unmanned  air 
vehicles  or  linked  UGVs,  as  communication  relay  stations. 
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4.0  MISSION  RELATED  FUNCTIONALITIES  AND 

TECHNOLOGIES 


The  purpose  of  this  section  is  to  make  comments  and  recommendations  on  technology 
issues  related  to  mission  areas.  The  major  emphasis  of  this  report  has  been  on  navigation 
related  technologies.  However,  in  the  process  of  this  investigation  it  became  apparent  there 
are  several  mission  issues  which  have  a  significant  impact  on  navigation  technologies  and 
their  utilization. 

Potential  missions  currently  envisioned  for  UGVs  include: 

•  Reconnaissance  Surveillance  and  Target  Acquisition 

•  Logistics 

-  Resupply  to  forward  areas 

-  Depot  management  of  material 

•  Security  surveillance 

•  Environmental  surveillance  (NBC  contamination) 

•  Weapons  platform  applications 

The  initial  battlefield  mission  that  will  be  addressed  by  UGVs  is  Reconnaissance, 
Surveillance  and  Target  Acquisition  (RSTA).  Logistics  is  also  being  addressed,  but  Demo 
I  and  Demo  II  are  both  initially  aimed  at  RSTA. 

An  initial  emphasis  on  navigation,  communications  and  RSTA  is  a  sound  development 
strategy,  since  these  functionalities  are  fundamental  to  any  battlefield  mission.  Even 
individual  soldiers  perform  RSTA  and  need  communication  as  part  of  their  basic  war 
fighting  capability;  a  UGV  in  a  danger  zone  can  do  no  less.  Furthermore,  UGVs  must  be 
able  to  move  and  operate  jointly  with  maneuver  forces  no  matter  what  basic  mission  it  is 
assigned. 

4.1  RSTA 

The  issues  related  to  RSTA  are  the  sensor  suite  and  the  associated  processing  required  to 
discriminate  targets  from  a  cluttered  background.  A  large  body  of  work  in  Automatic 
Target  Recognition/Automatic  Target  Cueing  (ATR/ATC)  could  potentially  be  applied  to 
this  area,  once  the  core  functionalities  have  been  satisfactorily  addressed.  The  basic 
sensors  for  RSTA  will  likely  be  different  than  sensors  for  navigation.  Thus,  the 
processing  algorithms  for  ATR/ATC  will  be  different  but  will  utilize  some  of  the  same 
techniques. 
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The  major  effort  being  expended  in  the  current  DOD  program  is  in  platform  mobility  to 
move  the  mission  sensors  on  the  batdefield.  However,  the  real  interest  of  the  users  will  be 
on  the  data  from  the  mission  sensors.  This  aspect  of  the  problem  must  be  given  more 
attention  as  we  progress  to  a  useable  system.  Automatic  Target  Recognition  (ATR)  and 
sensors  for  ATR  are  a  separate  specialty  and  are  being  worked  on  by  other  groups  in  DOD. 
It  is  not  desirable  to  duplicate  their  efforts.  However,  we  need  to  give  this  aspect  of  the 
problem  enough  effort  to  ensure  we  select  a  suite  of  sensors  which  will  fulfill  mission 
requirements  in  the  demonstration.  Harry  Diamond  Laboratories  is  addressing  this  aspect 
of  the  problem  in  Demo  I. 

The  Reconnaissance,  Surveillance,  and  Target  Acquisition  (RSTA)  mission  does  not 
require  negotiating  terrain  in  itself,  only  being  capable  of  getting  to  possibly  isolated  but 
specified  locations.  Therefore  a  vertical  takeoff  and  landing  system  to  carry  sensors  and 
equipment  to  these  locations  may  also  fulfill  this  mission.  This  system  could  get  to  more 
isolated  locations  quicker.  Such  a  system  has  been  developed  and  demonstrated  as  a 
sentinel  CL- 227  UA  V  by  Indal  Technology  of  Mississauga,  Ontario,  Canada  One 
fundamental  question  that  needs  to  be  answered  is  -  What  are  the  tradeoffs  between  getting 
the  sensors  to  location  in  the  air  versus  on  the  ground.  If  the  objective  is  to  assist  in  the 
development  of  autonomous  vehicle  technology  for  commercial  highways,  DOE,  NASA 
and  other  military  applications,  we  want  to  remain  on  the  ground.  Also,  transition  from  the 
first  generation  teleoperated  system  is  more  straight  forward  with  a  ground  system.  Noise 
and  payload  are  basic  questions  for  airlifted  system  versus  communication  necessary  for 
navigation  for  a  ground  system.  In  fact,  a  small  force  of  small  portable  sensing  systems 
containing  a  battety,  acoustic  sensor,  and  camera  to  supplement  the  larger  mobile 
autonomous  sensing  systems  may  be  desirable.  These  sensors  could  be  placed  near  roads 
or  strategic  areas  to  provide  additional  surveillance  information. 

Another  major  issue  that  needs  to  be  addressed  is  cheap,  lightweight  onboard  stabilization 
of  sensors. 

4.2  LOGISTICS 

Other  applications  of  the  technology  developed  herein  include  the  logistics  areas  of 
resupply  of  forward  areas,  depot  management  and  convoys  of  equipment/weapons.  These 
systems  utilize  some  of  the  technologies  developed  in  the  current  UGV  program,  but  will 
also  require  some  additional  developments. 

Resupply  of  forward  areas  requires  communications  with  both  the  front  units  and  the 
logistic  network.  Effector  technology  for  material  transfer  will  be  an  important  issue  which 
is  not  present  in  the  RSTA  mission.  Because  of  the  hazardous  nature  of  the  environments 
in  which  these  systems  will  operate,  rapid  and/or  automatic  material  transfer  while  moving 
will  be  desirable. 

Depot  management  and  material  distribution  at  various  levels  are  another  mission  for 
UGVs.  This  would  utilize  more  capable  UGVs  than  automated  warehousing  but  is  based 
on  a  similar  utilization  concept  This  would  reduce  personnel  demands  and  material  losses 
by  restricting  access  to  stores.  The  entire  process  of  material  acceptance,  handling, 
storage,  security,  inventory  management  and  redistribution  could  be  automated.  This  may 
require  electronic  labeling  of  cartons  and  active  interrogation  capability  be  added  to  the 
UGV.  . 
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The  success  of  road  following  demonstrated  by  CMU  should  be  followed  up  with  some 
useful  and  accomplishable  Advanced  Technology  Transition  Demonstration  that  could  lead 
into  a  fieldcble  automated  road  following  moduie  system.  Convoy  supply,  in  which 
vehicles  can  automatically  resupply  front  line  troops  along  specifically  defined  roads,  is  one 
potential  application.  These  systems  could  be  in  kit  form  which  would  reduce  the 
personnel  demands  for  some  logistics  applications.  There  are  other  transportation  or 
maneuvering  applications  which  could  also  be  satisfied  by  a  road  following  capability. 

4. 3  SECURITY  AND  ENVIRONMENTAL  SURVEILLANCE 

Security  and  environmental  sensing  (SES)  requires  additional  monitoring  sensors  for 
detecting  threats,  concentration  of  chemical  species,  and  radiation  levels.  Automating  these 
applications  with  UGVs  both  reduces  manpower  demands  and  removes  humans  from 
hazardous  locations.  Since  this  mission  is  similar  to  RSTA,  only  integration  of  these 
mission  sensors  into  the  UGV  should  be  required.  Repetitive  patrolling  of  security 
perimeters  is  a  logical  first  application  of  unmanned  ground  vehicles. 

4.4  WEAPONS  PLATFORM  APPLICATIONS 

Certain  weapon  platforms  are  used  in  a  repetitive  mode  and  are  hazardous  locations  as  they 
are  natural  high  priority  targets.  Utilizing  UGV  technology  for  this  application  has 
advantages  because  the  vehicle  can  be  designed  and  built  without  regard  to  crew 
requirements,  communication  can  be  automated  with  command  and  control,  and  fire  control 
can  be  automated  while  stationary  or  moving.  This  will  require  automated  weapon 
effectors  for  aiming,  loading,  and  firing  in  addition  to  the  basic  UGV  technologies 
currently  in  development. 
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5.0  ONGOING  WORK  AND  INSTITUTIONS 
ENGAGED  IN  TECHNOLOGY  DEVELOPMENT 

The  nation's  UGV  R&D  activities  are  currently  focused  on  three  major  areas.  These  are: 

•  DOD  Robotics  (UGV)  program  in  support  of  land  warfare, 

•  DOE  Environmental  Reclamation  and  Waste  Management  Program,  and, 

•  NASA  Space  Exploration  Initiative. 

These  R&D  activities  are  described  briefly  below. 

5 . 1  DOD  ROBOTICS  (UGV)  PROGRAM 

This  program  emphasizes  the  development  of  those  artificial  intelligence  and  robotics 
technologies  which  have  the  best  promise  as  force  multiplier  to  counter  the  effect  of  land 
warfare  reductions  as  well  as  provide  intermediate  teleoperated  UGV  solutions  that  remove 
the  soldiers  from  extremely  dangerous  activities  and  prepare  the  forces  for  future 
introduction  of  robotic  UGVs.  The  DOD  programs  consist  of  two  components:  advanced 
system  developments  and  technology  development  and  demonstrations. 

5.1.1  Advanced  System  Developments 

These  developments  address  near  and  mid-term  military  operational  requirements,  with 
initial  operational  capabilities  in  the  late  90‘s.  There  are  five  systems  currently  in  advanced 
development,  three  which  are  in  full  scale  advanced  development,  one  in  the  initial  stages 
of  entering  advanced  development  and  one  in  the  planning  stages.  These  will  be  described 
from  the  most  developed  to  the  more  preliminary  systems. 

5. 1.1.1  Remote  Ordinance  Neutralization  Device  (ROND) 

This  system  is  designed  to  provide  a  safe  remote  explosive  ordinance  handling  and 
disarming  capability.  This  system  when  fielded  in  the  late  90s  will  replace  an  assemblage 
of  service-specific  teleoperated  systems  that  have  very  limited  operational  and  utilitarian 
capabilities.  This  new  system  will  provide  EOD  personnel  the  means  to  secure  unexploded 
ordinance,  attache  a  render  safe  procedure  device/tool,  withdraw  to  a  safe  area,  and 
fire/function  the  tool.  The  ROND  platform  will  separate  the  EOD  operator  safely  from 
hazardous  accident/incident  sites  where  explosive,  chemical,  and  radiation  hazards  are 
present. 

The  ROND  consists  of  an  operator  control  console,  a  mobile  platform,  a  closed-circuit 
color  television  system,  an  RF  link,  a  tethered  link,  a  self-supporting  power  source,  and  a 
manipulator  consisting  of  an  arm  and  end  effector  with  common  tools.  The  control  console 
includes  an  RF  transmitter,  fiber-optic  transceiver,  video  receiver,  television  display 
monitor,  power  converter,  and  associated  electronics  for  mobile  platform  control  and 
telemetry.  The  mobile  platform  is  equipped  with  a  removable  closed-circuit  television 
camera  (with  ability  for  both  color  or  low-light  black  and  white)  with  zoom  lens,  removable 
video  lighting,  fiber-optic  and  RF  communication  links,and  associated  electronics.  The 
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fiber  tether  enables  system  commands  and  sensor  data  to  be  passed  between  the  mobile 
platform  and  control  console  when  RF  communication  is  not  possible  or  not  desired.  A 
lightweight  diesel  systems  will  supply  power  for  operational  functions.  The  ROND  is 
operable  in  all  expected  weather  conditions  including  rain,  snow,  sleet,  sand,  and  dust. 

The  platform  is  designed  for  low  velocity  driving  in  a  wide  spectrum  of  operational 
environments. 

5.1.1 .2  Rapid  Runway  Repair  (RRR1 

The  Rapid  Runaway  Repair  unmanned  ground  vehicle  is  a  remotely  operated  airfield 
restoration  UGV.  The  system  is  capable  of  runway  and  surface  assessment,  hole  refilling, 
debris  removal  and  surface  preparation. 

The  Rapid  Runway  Repair  UGV  provides  a  robotic  means  of  executing  runway  repair  and 
recover  including  cleanup  of  unexploded  ordinance,  and  thus  alleviates  some  of  the  safety 
and  manpower  issues  associated  with  RRR.  The  repair  vehicle  can  perform  under  post¬ 
attack  conditions  while  the  remote  operators  and  other  airbase  personnel  remain  at  a  safe 
distance.  Based  on  preliminary  performance  tests  of  a  brassboard  design,  it  is  estimated 
that  these  remotely  operated  RRR  machines  working  alone  will  decrease  crater  repair  time 
by  35  percent  compared  to  manually  operated  systems. 

5. 1 . 1 .3  Tactical  Unmanned  Ground  Vehicle  (TUGV) 

The  Tactical  Unmanned  Ground  Vehicle  program  is  a  joint  Army  -  Marine  Corps  project 
for  teleoperational/teleassisted  battlefield  RSTA. 

The  long  term  objective  of  the  TUGV  program  is  to  extend  the  operational  capability  to 
perform  RSTA-related  missions,  without  introducing  limitations  that  are  not  present  in 
alternative  manned  systems.  Future  TUGVs  will  allow  an  operator  to  control  multiple 
vehicles  and  oversee  several  missions.  Other  TUGV  characteristics  such  as  mobility, 
deployability,  and  endurance  should  match  or  exceed  those  of  comparable  manned 
systems.  The  program  will  increase  the  effectiveness  and  survivability  of  combined  armed 
forces  by  extending  the  control  radius  of  human  presence  on  the  battlefield. 

The  first  generation  TUGV  will  be  a  teleoperated,  lightweight,  helicopter  transportable, 
mobile  system.  Under  the  control  of  a  safely  positioned  remote  operator,  it  can  be 
navigated  to  its  position  and  perform  forward  observation  over  prolonged  time  periods.  In 
achieving  forward  presence,  this  system  will  significantly  lessen  the  exposure  of  combat 
soldiers  and  marines  to  hazardous  and  lethal  environments.  Potential  TUGV  missions 
include  RSTA  with  laser  designations  and  weapon  targeting;  NBC  detection  and 
surveillance;  and  obstacle  detection  and  breaching. 

5.1. 1.4  Exterior  Physical  Security  (EPS) 

This  project,  currently  in  the  initial  phases  of  starting  Advanced  System  Development  is  for 
guided/supervised  robotic  security  surveillance  and  enforcement  at  military  sites.  The  Joint 
Program  Office  for  Physical  Security  is  initiating  advanced  system  development  building 
on  technologies  developed  by  Demo  I  and  Sandia.  Because  of  the  constrained  environment 
in  which  this  UGV  will  operate,  it  has  the  potential  for  increased  robotization  using  rote 
robotic  technologies  (CARD,  Retrance,  Retrack,  etc.)  A  major  remaining  issue  is  which 
technologies  are  affordable. 
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5. 1.1.5  Robotic  Countermine  fRCMi 


This  project  is  currently  in  early  planning  to  exploit  guided/rote  robotic  technologies  for 
mine  detection  and  clearing.  Currently  OSD  is  evaluating  technologies  available  for 
protecting  our  maneuvering  forces  from  land  mines  and  looking  at  both  near  and  far  term 
options.  The  current  emphasis  is  how  we  can  adapt  fielded  armored  combat  vehicles  for 
the  countermine  mission. 


5.1.2  Technology  Development  and  Demonstration 
5. 1.2.1  Demo  I 

This  activity  has  been  in  progress  for  several  years  and  will  culminate  with  several 
organizations  participating  in  a  performance  demonstration  scheduled  for  the  last  week  of 
April  and  the  first  week  of  May.  The  objective  is  to  demonstrate  the  current  state  of 
teleoperated  and  teleassisted  robotic  technologies  to  the  user  community. 

Six  HMMWVs  developed  by  different  laboratories  or  Commands  will  be  used  in  the 
demonstration  (one  from  BRL,  one  from  HDL,  one  from  1ST  and  three  from  TACOM). 
They  will  be  operated  remotely  via  RF  link  but  will  include  a  demonstration  of  a  variety  of 
teleassistance  technologies  such  as  retrotraverse.  Operator  control  systems  (both  portable 
and  the  multiple  vehicle  control  system)  will  be  part  of  the  evaluations.  Various  navigation, 
reconnaissance,  surveillance,  and  targeting  sensors  and  displays  are  incorporated  on  the 
six  different  vehicles  in  order  to  demonstrate  their  effectiveness  on  unmanned  vehicles  in 
battlefield  scenarios. 

The  principal  purpose  of  Demo  I  is  to  mature  critical  system  component  technologies  for 
first  generation  UGVs  and  demonstrate  their  readiness  for  acquisition  programs.  Based  on 
the  results  of  Demo  I,  selected  technologies  will  be  integrated  into  the  basic  STV  for  the 
development  of  a  full-up  TUGV  prototype.  The  emphasis  is  on  reducing  operator 
workload  while  enhancing  performance  of  the  RSTA  mission.  Demo  I  includes  products 
resulting  from  non-UGV  program  activities  in  order  to  maximize  technology  transfer  to  the 
UGV  program.  The  majority  of  the  hardware  and  software  for  Demo  I  has  already  been 
developed  and  is  operational. 

As  described  below.  Demo  I  emphasizes  integrated  testing  and  technology  maturation 
demonstration  in  the  five  critical  technology  areas: 

•  Human  Factors/Man-Machine  Interface  -  Operator  workload  and  performance  for 
the  teleoperated  and  supervisory  modes  of  operation. 

•  Communication  -  The  relative  utility  of  various  wide-band  communications  and 
low  data-rate  communication  alternatives. 

•  Navigation  -  The  effects  of  high  and  low  data  rate  operations  on  full-time 
teleoperated  and  supervisory  control  for  day  and  night  mobility  operations. 
Automated  mobility  modes  such  as  retrotraverse,  and  obstacle  detection/avoidance. 

•  RSTA  Mission  Package  -  Mission  package  performance  while  the  vehicle  is 
stationary. 
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•  Computer  Control  Architecture  -  New  options  for  UGV  control  through  integration 
of  teleoperated  and  limited  robotic  control  modes. 

2l122J2£ML  IJ 

The  program’s  objective  is  to  develop  and  mature  within  five  years  those  navigation 
technologies  which  are  critical  for  the  system  development  of  supervised  autonomous 
unmanned  ground  vehicles  to  meet  the  needs  of  the  ground  forces  of  the  future.  Program 
focus  is  on  exploiting  the  artificial  intelligence  advances  made  by  DARPA's  science  and 
technology  program.  The  developed  navigational  technologies  will  be  transitioned  to  the 
principal  DoD  agencies  which  are  responsible  for  and  support  the  acquisition  of  unmanned 
ground  vehicles  and  to  industry  to  establish  an  R&D  base  that  will  support  their 
development 

The  purpose  of  Demo  II  is  to  demonstrate  those  navigation  technologies  that  are  critical  to 
evolving  UGVs  from  labor  intensive  teleoperation  to  supervised  autonomy.  These 
technologies  will  enable  TUG  Vs  to  operate  in  a  limited  bandwidth,  tactical  communications 
environment  by  incorporating  a  cost-effective,  semi-autonomous  navigation  system, 
which  removes  the  requirement  for  the  UGV  operator  to  maintain  continuous  control  of 
driving  and  RSTA  functions.  Program  emphasis  is  on  exploiting  emerging  hardware  and 
software  advances  in  passive  and  active  sensing,  autonomous  navigation,  and  high 
performance  computing  and  in  demonstrating  their  maturity  for  acquisition  programs  of 
second  generation  UGVs  by  1995. 

Demo  II  is  carried  out  through  a  four  phase  demonstration  directed  approach,  successively 
increasing  the  complexity  of  the  UGV  system  and  its  robotic  capabilities.  Each  phase  is 
executed  in  two  steps.  First,  the  technologies  to  be  demonstrated  will  be  integrated  by  the 
participating  universities  into  a  breadboard  system  on  a  HMMWV  at  three  geographical 
sites  (CMU,  U  of  Mass.,  and  Stanford).  After  their  performance  has  been  validated,  the 
technologies  will  be  transferred  to  Martin  Marietta  Corp.  for  a  brassboard  system 
integration. 

The  systems  integrator  and  major  contractor  for  this  activity  is  Martin  Marietta  -  Denver. 
They  will  be  responsible  for  the  integration  of  technologies  being  developed  at  several 
institutions  described  below.  The  principal  participating  organizations  and  their  major  areas 
of  activity  are  as  follows: 

Organization  Activity 


Martin  Marietta  -  Denver 
Carnegie  Mellon  University 

Stanford  Research  Institute 

NASA  Jet  Propulsion  Laboratory 


System  Integration 

System  Development,  Perception, 
Road  Following 

Perception,  Stereo  Vision 
Terrain  Understanding 

Perception,  Stereo  Vision 
Micro-robots 
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University  of  Massachusetts 
University  of  Maryland 
ADS 

Hughes  Research  Laboratories 

University  of  Michigan 

Odetics 

Alliant 


Perception,  Landmark  Recognition, 
IUA 

Perception,  Vision-based  Navigation 
and  Recognition 

Planning,  Mission  Planning 

Planning,  Path  Planning,  Behavior 
IUA 

Cooperative  Vehicles 

Sensors 

Sensors 


The  activity  and  responsibility  of  these  groups  have  just  been  defined  and  contracts  have 
been  negotiated.  Following  is  a  brief  summary  of  the  activities  in  which  these 
organizations  participate. 

Carnegie  Mellon  has  been  very  active  in  the  development  and  demonstration  of  autonomous 
vehicles  for  the  past  several  years.  They  have  operated  both  Navlab  I  and  Navlab  II  in 
several  autonomous  demonstrations.  They  have  developed  perception  and  road  following 
algorithms  which  will  be  incorporated  as  part  of  the  demonstrations. 

Stanford  Research  Institute  is  conducting  research  in  the  areas  of  perception  and  stereo 
vision.  They  have  been  active  in  developing  design  curves  to  define  the  limits  of  stereo 
vision  as  well  as  metrics  for  vision  systems.  Stereo  analysis  techniques  developed  by  SRI 
will  be  used  to  provide  understanding  for  practical  application  of  stereo  vision. 

NASA  J PL  is  active  in  developing  real  time  stereo  vision  techniques,  perception,  and 
micro-robotics  for  demonstrating  rough  terrain  navigation.  They  are  hlso  developing 
obstacle  detection  techniques  and  implementing  them  on  a  HMMWV  capable  of  operating  at 
10  mph. 

The  University  of  Massachusetts  at  Amherst  are  working  on  perception,  landmark 
recognition  and  a  processor  for  high  speed  image  analysis  (Image  Understanding 
Architecture,  IUA).  The  perception  and  landmark  recognition  work  is  geared  to  the  goal  of 
autonomous  navigation.  This  work  is  to  develop  an  autonomous,  outdoor  mobile  UGV 
capable  of  accomplishing  real  time  mobile  robot  navigation  in  a  variety  of  non-trivial 
outdoor  domains.  The  Image  Understanding  Architecture  prototype  development  has  the 
objective  of  developing  a  three  level  parallel  processing  prototype.  Other  companies 
working  on  this  project  include  Hughes  and  Amerinex  Artificial  Intelligence  (AAI). 

ADS  is  working  on  path  planning  and  mission  planning  for  Demo  D.  They  are  working  on 
case  based  planning  and  qualitative  navigation.  Their  planning  approach  involves  storing 
high  level  plans  for  specific  conditions,  using  model  based  techniques  for  planning  on  the 
fly  and  using  a  scheme  derived  from  their  work  with  resource  allocation  to  handle 
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conflicting  goals.  Qualitative  navigation  is  a  method  of  path  planning  using  incomplete 
information  of  landmarks.  They  began  contract  work  in  December  1991  so  they  are  just 
beginning  to  develop  their  approach  and  activities  for  Demo  II. 

Hughes  Research  Laboratories  is  working  on  vehicle  path  planning  and  behavior.  Their 
approach  includes  making  more  information  available  for  continuous  planning  during 
motion  by  using  gradient  field  and  higher-dimensional  representations.  They  plan  to 
provide  arbitration  by  developing  a  scheme  in  which  multiple  low  level  behaviors  are  acti  • 
in  parallel  and  each  rates  the  desirability  of  specific  actions  with  respect  to  that  behavior. 

In  addition,  Hughes  is  doing  the  hardware  for  the  Image  Understanding  Architecture.  This 
paralLl  processor  is  expected  to  be  ready  by  mid- 1992  and  delivered  at  that  time  to  the 
other  participants  in  this  project  (AAI  and  the  University  of  Massachusetts). 

The  University  of  Michigan  is  working  on  cooperative  techniques  for  multiple  vehicles.  In 
their  approach,  called  partial  global  planning,  local  detailed  plans  are  formulated  for  each 
vehicle  based  on  local  conditions  and  the  mission  of  the  vehicle.  These  local  plans  are  th 
abstracted  and  communicated  to  the  other  vehicles  in  the  formation.  Thus,  communication 
bandwidth  is  reduced  and  commitment  to  detail  is  avoided. 

Odetics  and  Alliant  are  working  on  3-D  sensors  for  vehicle  navigation.  These  are  active 
laser  ranging  systems  for  navigation. 

5.2  DEPARTMENT  OF  ENERGY 

Unmanned  ground  vehicle  research  at  the  Department  of  Energy  is  located  at  Oak  Ridge 
and  Sandia  National  Laboratories.  Activity  is  focused  on  teleoperations  and  on  mobile 
robot  navigation  in  confined  spaces.  The  teleoperation  research  is  focused  on  nuclear  fuels 
processing  and  the  mobility  work  is  for  remote  vehicle  operation  for  security  applications 
and  hazardous  material  sensing  and  cleanup.  The  DoE  and  DoD  programs  in  this  area  are 
interrelated  and  coordinated  (see  section  7.8  for  more  detail). 

5.3  NASA-JPL 

JPL  is  conducting  research  on  techniques  for  real  time  stereo  image  registration  and 
perception  from  stereo  vision.  Additionally  they  have  developed  large  and  small  robots  for 
implementing  and  testing  mobility  and  navigation.  They  have  developed  a  six  wheeled 
twenty  five  hundred  pound  robot  (Robby),  a  six  wheeled  fifty  two  pound  robot  (Rocky  3) 
and  they  are  conducting  research  with  MIT  on  micro  robots.  Under  contract  to  the  U.S. 
Army  Tank-Automotive  Command,  JPL  has  developed  the  Robotic  Technology  Test 
Vehicle.  This  vehicle,  and  its  associated  van  for  remote  command  and  control,  allow  a 
human  operator  to  remotely  operate  the  vehicle  over  a  reduced-bandwidth  communication 
channel.  This  is  accomplished  by  means  of  Computer-Aided  Remote  Driving  (CARD) 
technology  developed  at  JPL,  where  the  human  operator  views  static  stereo  imagery 
returned  from  the  vehicle  and  plans  a  safe  path  for  the  vehicle  to  follow  as  far  ahead  as  can 
be  seen  (see  section  7. 10  for  more  detail). 
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6.0  FINDINGS  AND  RECOMMENDATIONS 

This  study  was  a  review  and  analysis  of  research  and  development  ongoing  and  required 
for  autonomous  ground  vehicle  navigation,  and  this  section  contains  a  summary  of 
observations  from  this  study  and  issues  for  consideration  and  analysis.  The  projects 
ongoing  at  DOE  and  NASA  are  synergistic  and  coordinated  with  DOD  so  duplication  of 
effort  is  not  a  concern.  Also  there  are  no  glaring  deficiencies  in  the  coverage  of 
technologies  required  for  autonomous  navigation.  However,  robustness  and  time  to 
develop  are  concerns  with  some  of  the  technologies  as  described  in  sections  6.1  and  6.2. 
General  findings  and  recommendations  are  included  in  6.1  through  6.4  and  specific 
recommendations  are  included  in  6.5. 

6.1  TERRAIN  TRAVERS  ABILITY 

Demo  II  is  addressing  perception  planning  at  several  levels,  sensors  for  navigation, 
algorithms  for  obstacle  avoidance,  and  processors  required  for  real  time  control.  A  real 
problem  in  autonomous  navigation  in  the  battlefield  area  is  traversability  of  unknown  areas 
(i.e.,  how  solid  is  the  ground?,  how  deep  are  the  holes?,  how  steep  is  the  slope?,  etc). 
These  problems  need  to  be  addressed  before  selection  of  the  sensor-processor  suite  for 
navigation  in  order  to  ensure  the  ones  selected  have  the  required  capability.  Traversability 
of  areas  that  have  not  been  seen  previously  by  the  local  force  structure  is  very  difficult  and 
may  require  either  detailed  premapping  of  the  area  of  operation  by  remote  (airborne  or 
satellite)  assets  or  new  sensors  capable  of  detecting  ground  water  content  or  soil 
composition.  This  will  become  a  concern  as  initial  autonomous  navigation  capabilities  are 
demonstrated  and  robustness  of  the  capability  comes  into  consideration. 

We  recommend  that  a  project  to  specifically  address  this  problem  be  initiated.  We  believe 
this  will  require  a  specially  designed  new  sensor  but  recommend  the  project  include  a 
requirements  analysis  phase  and  an  options  analysis  phase  before  a  sensor  design  and 
construction  phase  is  initiated. 

6.2  INTEGRATED  SENSOR-PROCESSING  ANALYSIS 

To  achieve  fully  autonomous  navigation  from  a  scientific  viewpoint,  integrated  research 
should  be  pursued  more  than  has  been  present  to  date.  Automated  planning  (including 
obstacle  avoidance)  is  ultimately  based  on  what  information  is  provided  from  available 
sensors.  Present  planning  algorithms  make  use  only  of  what  existing  modalities  provide 
them.  Some  sense  of  the  reverse  must  or  should  occur.  Currently,  no  sensor  research  is 
planned  based  on  the  information  needs  of  planning  algorithms.  This  may  lead  to  a 
situation  where  the  most  optimum  approach  is  not  followed.  In  any  sensor-processing 
application,  there  is  always  a  fundamental  trade-off  between  capabilities  (smarts)  in  the 
sensor  and  capability  (smarts)  in  the  processing  algorithms  and  processor.  This  tradeoff 
needs  to  be  very  carefully  examined  for  the  battlefield  autonomous  navigation  application. 

A  new  approach  to  development  would  be  to  compile  a  list  of  needed  environmental 
qualities  (such  as  traversability,  detecting  holes  and  depressions,  vegetation,  range  maps, 
...),  determine  if  existing  sensing  modalities  can  provide  this  information  and  make 
recommendations  regarding  new  modalities  if  such  are  required. 
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6 . 3  MULTIPLE  APPROACHES 

The  DEMO  II  project  will  address  stereo  vision  and  3-D  range  finding  for  navigation 
sensing,  several  planning  and  obstacle  avoidance  approaches,  and  iWarp  and  IUA  for  the 
processor.  Multiple  approaches  help  reduce  risk  and  ensure  success  in  a  research  project. 
However,  there  is  a  desire  on  the  part  of  the  users  not  to  accept  the  vulnerability  of  active 
systems  in  a  hostile  environment.  The  tradeoff  between  vulnerability  for  active  systems 
and  complexity,  range,  and  accuracy  for  stereo  systems  have  many  implications  in  cost, 
logistics,  reliability .  and  other  implementation  issues.  We  need  to  be  certain  we  consider 
these  multiple  factors  in  the  selection  process  between  multiple  approaches  and  not  select 
approaches  based  solely  on  maturity  of  development.  We  recommend  this  question  be 
addressed  now  to  ensure  we  really  have  the  options  we  are  pursuing. 

6.4  UGV  ACQUISITION  STRATEGY 

The  concept  of  introducing  a  sequence  of  a  small  quantity  of  teleoperated  vehicles  into  the 
force  structure  and  gradually  providing  increased  autonomous  capability  is  excellent.  It 
would  not  be  possible  to  suddenly  introduce  an  autonomous  vehicle  into  the  force. 
Introducing  teleoperated  vehicles  will  allow  their  integration  into  the  command  structure 
and  the  forces  to  become  familiar  with  the  potential  utility  of  such  a  system.  They  will  also 
be  able  to  provide  valuable  feedback  to  developers  of  the  next  generation  designs. 

6.5  FUNCTIONAL  DEVELOPMENT  STATUS 

The  findings  and  recommendations  regarding  the  state  of  development  of  the  various 
functionalities  needed  for  autonomous  navigation  are  presented  by  function  in  this  section. 

6.5.1  Navigation  Sensing 

Current  capabilities  of  sensing  modalities  are  not  adequate  for  the  higher  levels  of 
autonomy,  and  there  is  some  question  that  the  modalities  themselves  will  ever  be  fully 
adequate  for  navigation  sensing  for  supervised  or  autonomous  vehicles.  New  modalities 
should  be  explored. 

The  search  for  new  modalities,  or  the  refinement  of  existing  modalities,  needs  to  be  driven 
by  an  overall  systems  approach  that  seems  to  be  lacking  in  current  UGV  research  and 
development.  Each  development  community  (sensor,  planning,  communications,...) 
seems  to  be  deriving  requirements  independent  of  each  other.  This  seems  to  be  less  true  in 
sensors,  but  is  still  a  significant  factor. 

Specific  examples  can  be  pointed  to  where  an  integrated  approach  is  lacking.  For  instance, 
traversability  assessments  are  crucial  for  off-road  navigation,  even  when  the  human  is  in 
direct  control  of  a  vehicle  from  a  remote  location.  An  operator  or  automated  driving 
algorithm  will  need  to  know  if  the  patch  of  ground  ahead  of  the  unit  is  swampy  or  soft  or 
vegetation  is  covering  an  impassable  ditch  or  barrier.  Current  EO  and  active  sensors  cannot 
provide  this  kind  of  information,  nor  have  they  attempted  to.  Furthermore,  resolution,  area 
coverage  and  frame  rate  requirements  have  not  been  algorithm  driven. 

Sensing  modalities  that  could  be  explored  include  novel  passive  stereo,  active  and  passive 
polarization  exploitation,  active  and  passive  milli-meter  wave,  multi-coior/multi-wavelength 
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band  sensing  as  well  as  more  system-exotic  ideas  like  multi-sensor  fusion  for  synergistic 
exploitation  of  observables. 

To  date,  requirements  on  the  sensor  have  not  been  driven  by  what  is  required  by  the 
cognition  module.  The  issue  of  requirements  on  the  sensor  suite  driven  by  an  analysis  of 
what  is  required  by  cognition  module  has  not  been  fully  addressed. 

6.5.2  Navigation  Perception 

Navigation  perception  is  only  required  for  guided,  supervised,  and  autonomous  levels  of 
autonomy.  The  following  comments  apply  to  all  three  levels. 

•  Road  following  has  been  demonstrated  in  a  research  setting.  Within  this  report, 
this  is  considered  a  limited  subset  of  guided  robotics.  Given  the  success 
demonstrated,  road  following  could  be  implemented  in  an  advanced  system 
development  within  a  short  dme. 

•  Obstacle  detection  is  more  developed  than  detecting  holes  or  depressions  (even  for 
teleoperation). 

•  Sufficiently  fast  methods  (2-4  Hz)  which  produce  depth  maps  for  limited 
applications  from  stereo  camera  pairs  should  be  available  by  the  year  1994,  but 
accuracy  and  resolution  will  be  limited,  requiring  powerful  perception  algorithms 
to  understand  the  output  from  such  sensor  data.  Resolution  accuracy  is  on  the 
order  of  1-5  inches  of  spatial  resolution  in  order  to  detect  obstacles. 

Sensing  modalities  that  can  provide  consistent  and  complete  information  are  a  major  reason 
for  current  limitations  in  robotic  functionality.  Current  perception  algorithms  are  overtaxed 
and  slow,  not  because  of  fundamental  limitations  in  and  of  themselves,  but  because  there  is 
not  rich  enough  information  being  received  from  current  sensing  modalities.  If  this 
information  were  available,  the  problems  could  conceivably  be  entering  DEM/VAL  by 
1996. 

Again,  like  all  areas  of  robotics,  an  integrated  approach  to  defining  requirements  and 
developing  systems  should  gready  affect  the  development  of  perception  algorithms.  A 
greater  appreciation  of  sensing  modalities  and  their  potential  information  content  would 
positively  affect  the  development  of  these  algorithms. 

6.5.3  Navigation  Planning 

Autonomous  planning  applies  to  the  higher  levels  of  autonomy  (guided,  supervised,  and 
autonomous).  Path  planning,  local  planning,  and  reactive  planning/obstacle  avoidance  are 
all  being  researched  and  several  approaches  are  being  developed  (see  section  3.4).  It  is 
expected  these  approaches  will  yield  significant  results  by  1996. 

Mission  planning  for  UGV  applications  has  not  been  addressed  as  a  separate  field  of  study, 
however  it  may  not  need  to  be,  given  the  great  body  of  work  in  mission  planning  as  a 
whole.  Undoubtedly  subtleties  and  variations  on  the  main  thrust  of  mission  planning  will 
be  required  for  UGVs,  but  as  a  whole,  the  field  may  well  be  in  hand  for  the  lower  levels  of 
autonomy  up  through  supervised. 
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Autonomous  mission  planning,  as  we  have  defined  it,  has  not  been  addressed.  It  is 
expected  that  some  variant  of  a  gradient  field  approach  will  prove  useful.  However,  it  will 
likely  be  the  year  2000  before  an  acceptable  level  of  satisfaction  will  be  achieved. 

6.5  l  World  and  Environment  Modeling 

World  modeling  applies  only  to  the  higher  levels  of  autonomy  and  modeling  based  on 
Defense  Mapping  Agency  (DMA)  products  is  expected  to  be  developed  by  1996. 
Environment  models  are  more  difficult  than  world  models  and  are  required  for  some  types 
of  teleassistance  in  addition  to  guided,  supervised,  and  autonomous  levels  of  autonomy. 

Current  trends  in  environmental  modeling  will  solve  the  problems  of  the  higher  levels  of 
autonomy  in  the  projected  future.  Occupancy  maps  with  information  "tagged"  inside  cells 
appear  to  have  the  brightest  future.  The  greatest  challenge  will  be  in  the  fully  autonomous 
field,  since  so  much  information  must  be  encapsulated  and  be  accessed  in  short  periods  of 
time. 

6.5.5  Communications 

The  communication  technologies  required  for  military  application  of  UGVs  are,  for  the 
most  part,  currently  available,  hence  the  projected  date  of  1992  shown  for  this  functionality 
in  Figure  8.  The  challenges  arise  from  the  superposition  of  military  operational 
requirements  on  the  UGV  communication  equipment  These  challenges  are  particularly 
acute  at  the  lower  levels  of  UGV  autonomy  which  require  continuous  communication  for 
effective  operation  (from  a  navigation  standpoint).  To  reduce  the  burden  which  continuous 
communication  imposes  at  the  lower  UGV  levels,  the  main  thrust  of  near-term  research 
should  be  data  compression  technologies.  This  thrust  should  include  compression  efforts  in 
both  data  processing  and  transmission  techniques.  It  is  believed  that  this  technology  will 
continue  to  be  useful  as  the  UGV  platform  evolves  to  a  more  intelligent  mechanism  The 
level  of  support  should  reflect  the  fact  that  this  area  already  enjoys  significant  worldwide 
interest  and  investment.  Hence,  we  recommend  that  continuing  efforts  in  support  of  UGV 
objectives  include  evaluation  of  new  hardware  and  software,  as  they  appear,  and  the 
RDT&E  of  component  technologies  which  are  specific  to  UGV  needs.  We  further 
recommend  that  the  organizations  involved  in  UGV  video  compression  RDT&E  be 
encouraged  to  communicate  and  identify  synergistic  opportunities. 

Additionally,  technology  that  supports  beyond  line-of-sight  operation,  without  a  physical 
tether,  needs  to  be  explored  further.  Examples  include  the  use  of  satellites,  unmanned  air 
vehicles  or  linked  UGVs,  as  communication  relay  stations. 
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7.0  DOCUMENTATION  AND  VISIT  SUMMARIES 


7.1  FMC 

The  robotic  projects  we  reviewed  at  FMC  are  mainly  conducted  under  contract  from 
TACOM.  These  projects  are  focused  primarily  on  satisfying  the  needs  of  Demo  I.  FMC  is 
using  four  HMMWV  vehicles  for  this  effort;  two  from  FMC  and  two  from  the  previous 
TEAM  program.  FMC  has,  in  the  past,  developed  the  MVCT  testbed  (Multiple  Vehicle 
Control  Testbed),  which  was  derived  from  the  Robotic  Command  Cv.ner,  both  TACOM 
programs.  FMC  is  also  incorporating  JPL's  CARD  system  into  the  testbed.  This  effort 
includes  working  with  ADS  to  incorporate  TeamWorks  into  their  programs.  The  FMC 
program  manager  is  Lou  McTamaney  and  the  government  program  monitor  for  FMC’s 
efforts  is  Bruce  Brendell,  who  resides  at  TACOM. 

FMC  has  also  developed  a  high  speed  chip  for  route  planning  and  a  Terrain  Analysis 
Processor  for  use  in  the  command  center.  FMC  has  access  to  global  planning  software 
from  a  number  of  contractors  through  their  role  as  integrator  on  the  Advanced  Ground 
Vehicle  Technology  program  (a  program  to  increase  automation  of  manned  combat 
vehicles)  and  other  programs.  FMC  developed  a  communication  protocol  for  use  in  Demo 
I  which  they  feel  is  of  general  utility  for  controlling  multiple  unmanned  vehicles. 

Other  research  at  FMC  includes  porting  algorithms  for  recognizing  landmarks,  extracting 
road  information  and  understanding  ERIM  laser  scanner  images  to  a  16-node  butterfly 
processor  built  by  BBN,  Inc.  FMC  is  also  investigating  neural  net  technology  for 
autonomous  vehicle  vision  processing. 


Figure  9  illustrates  graphically  FMC’s  contributions  to  UGV  functionalities. 
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Figure  9.  FMC’s  Contributions  to  UGV  Functionalities 


7 . 2  STANFORD  UNIVERSITY 

Funding  for  UGV  related  efforts  at  Stanford  University  is  mainly  provided  by  NASA. 
Their  major  activities  are  investigating  methods  for  control  of  manipulators  on  mobile 
platforms  in  gravity-free  situations.  Stanford  has  demonstrations  of  capture  of  a  moving 
object  by  two  arms  mounted  on  a  single  moving  platform.  This  technology  is  potentially 
useful  in  downstream  applications  and  it  may  have  some  relevance  to  Demo  II,  if 
Reconnaissance,  Surveillance  and  Target  Acquisition  or  mine  neutralization  is  performed 
during  motion.  The  points  of  contact  at  Stanford  University  are  Tom  Vincent  and  Bob 
Cannon. 

Figure  10  illustrates  graphically  the  work  Stanford  is  performing  against  the  required 
functionalities  derived  in  Figure  7. 
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Figure  10.  Stanford’s  Contributions  to  Functionalities  Checklist  Derived 

From  Figure  7 


7 . 3  ADVANCED  DECISION  SYSTEMS  (ADS) 

ADS  is  the  second  planning  contractor  for  DARPA’s  Demo  II.  They  plan  to  pursue  two 
major  areas  of  research: 

1)  A  case-based  planner. 

2)  qualitative  navigation. 

In  addition,  they  will  look  at  cooperation  among  a  group  of  vehicles.  A  contract  for  this 
effort  was  recently  awarded  and  is  managed  by  Capt.  Koetz  at  TACOM. 

ADS  is  taking  a  case-based  approach  to  planning,  in  which  high-level  plans  are  stored  for 
specific  conditions.  They  intend  to  develop  model-based  techniques  for  planning  while 
moving.  ADS  expects  to  handle  conflicting  goals  by  prioritizing  actions  in  a  scheme 
derived  from  their  work  with  resource  allocation. 


ADS’  qualitative  navigation  work  is  published.  They  are  planning  to  extend  it,  but  have 
not  specified  exacdy  what  level  of  effort  would  be  extended.  Briefly,  qualitative  navigation 
is  a  method  of  path  planning  using  incomplete  information  of  landmarks.  Landmarks  are 
either  stored  or  learned  during  operation  and  stored. 

Cooperation  will  be  implemented  at  both  the  Operator  Control  Unit  level  (which  is  outside 
ADS’  scope)  and  at  the  vehicle  level.  ADS  is  planning  to  study  cooperation  with  and 
without  explicit  communication.  They  are  concerned  about  coordinating  the  cooperative 
planning  from  the  OCU  with  that  on  the  vehicles. 

Figure  1 1  illustrates  graphically  the  work  ADS  is  performing  against  the  identified 
technologies  required  for  UGV  implementation. 


Figure  11.  ADS'  Contributions  to  Functionalities  Checklist  Derived  From 

Figure  7 


7.4  NAVAL  POSTGRADUATE  SCHOOL 

Currently,  work  at  the  Naval  Postgraduate  School  is  being  performed  in-house  without 
outside  contractual  support  The  major  activities  involve  developing  mobile  robots  for 
indoor  environments,  although  they  will  soon  start  a  new  program  on  an  autonomous 
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underwater  swimmer.  During  ERIM's  visit,  they  discussed  work  on  resolving  ambiguities 
in  sonar  data  of  irregularly  shaped  rooms  and  structures  and  demonstrated  interesting 
studies  of  the  algorithm  accuracy.  They  have  also  developed  a  new  representation  of  turns 
and  curved  paths  which  minimizes  slippage  and  maintains  precise  positioning  information. 
This  research  seems  most  useful  as  a  source  of  some  specific  techniques  which  might  be 
applied  to  the  outdoor  UGV  problem.  The  points  of  contact  at  the  Naval  Postgraduate 
School  are  Yutaka  Kanayam  and  Robert  McGhee  of  the  Department  of  Computer  Science. 

Figure  12  illustrates  graphically  the  relevance  of  the  Naval  Postgraduate  School’s  work 
against  UGV  functionalities.  It  should  be  kept  in  mind  that  the  environment  that  these 
functionalities  are  applicable  to  at  the  Naval  Post  Graduate  School  are  significandy  different 
than  those  encountered  in  unmanned  ground  vehicle  applications. 


Levels  of  Autonomy 

Telepresence  Teleoperauon  Tele  assistance  Guided  Robotics  Supervised  Autonomous 


Figure  12.  Naval  Postgraduate  School  Contributions  to  Functionalities 

Checklist  Derived  From  Figure  7 

7.5  HUGHES  RESEARCH  LABORATORIES 


Hughes  is  one  of  two  principal  “planning”  contractors  for  DARPA’s  Demo  II  (the  other 
being  ADS).  The  effort  will  be  managed  by  CAPT  Koetz  of  TACOM,  acting  as  DARPA’s 
agent. 
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The  planning  approach  to  be  taken  by  Hughes  on  the  DARPA  program  is  based  on  their 
experience  on  ALV.  Basic  technical  issues  raised  by  that  experience  were: 

1)  command  arbitration  problems,  and 

2)  problems  introduced  by  abstraction  of  plans. 

In  addition  to  vehicle  planning,  Hughes  will  have  tasks  for  mission  pre-planning  and  a 
simulation  environment. 

Hughes  plans  to  address  the  problem  of  command  arbitration  by  developing  a  scheme  in 
which  multiple  low  level  behaviors  are  active  in  parallel  and  each  rates  the  desirability  of 
specific  actions,  such  as  a  turn,  with  respect  to  that  behavior.  By  this  means,  Hughes 
hopes  to  maintain  consistency  of  actions  with  multiple  goals,  e.g.  simultaneously  staying 
on  a  road  and  avoiding  obstacles. 

The  problems  associated  with  abstraction  of  plans  is  to  be  addressed  by  replacing  the  route- 
based  planning  developed  for  ALV  with  map-based  planning.  Hughes  is  considering  ways 
to  make  more  information  than  a  simple  path  description  available  for  continuous  planning 
during  motion.  Possibilities  include  new  representations  of  map-based  data,  such  as 
gradient  field  and  higher-dimensional  representations.  A  gradient  representation  associates 
a  direction  of  desired  motion  to  a  goal  with  each  point  in  a  map  and  is  produced  in  the  same 
process  that  would  be  used  to  derived  a  specific  path. 

In  addition  to  these  technical  problems,  Hughes  also  has  the  charter  to  develop  a  mission 
pre-planning  system  for  the  Demo  II  contractors,  intended  to  enable  an  operator  at  a  control 
station  to  lay  out  a  general  mission  plan  for  multiple  vehicles.  The  kinds  of  capability 
envisioned  included  3D  display  of  terrain  data,  simulation  of  vehicle  movement,  and 
mission  rehearsal.  A  major  issue  will  be  the  type  and  quality  of  data  available.  The  5m 
DTED  resolution  used  on  ALV  may  be  adequate  for  terrain  data,  and  the  DMA  TTD  and 
ITD  products  may  also  be  adequate.  A  vehicle  model  with  sufficient  detail  to  include 
response  times  and  distances  will  be  required  for  simulation.  As  part  of  the  pre-planning 
work,  Hughes  is  exploring  heuristic  methods  for  analyzing  visibility  at  locations  on  the 
terrain. 

Hughes  is  expected  to  provide  all  contractors  with  a  simulation  environment  in  which  to 
conduct  experiments. 

Figure  13  illustrates  which  identified  functionalities  the  Hughes  effort  is  contributing 
towards  enhancing  from  Figure  7. 
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Figure  13.  Hughes  Contributions  to  Functionalities  Checklist  Derived  From 

Figure  7 

7.6  UNIVERSITY  OF  MASSACHUSETTS,  AMHERST 

The  University  of  Massachusetts,  Amherst  is  pursuing  research  geared  towards  the  far  term 
goal  of  solving  the  problem  of  autonomous  navigation  in  terms  of  its  "base"  research  in 
machine  vision  and  image  understanding.  This  means  that  most  of  the  research  falls  in  the 
category  of  Perception  and  Cognition.  It  should  be  noted  that  this  group  has  not  aimed  at 
developing  technology  in  sensor  phenomenology,  but  is  aimed  at  processing  information 
gained  from  available  sensors.  Its  image  understanding  research  is  geared  towards  using 
flat  two  dimensional  images,  stereo  pairs  and  motion  cues  produced  from  electro-optical 
devices.  Research  at  UMass  also  includes  planning  for  action.  The  points  of  contact  at 
UMass  are  Professor  Ed  Riseman  and  Professor  A1  Hanson. 

Two  projects  in  support  of  DEMO  II,  both  DARPA  funded,  are  being  pursued  at  UMass 
currently.  They  are: 

*  Image  Understanding  and  Intelligent  Robot  Navigation 

•  Image  Understanding  Architecture. 

Image  Understanding  and  Intelligent  Robot  Navigation  is  a  program  where  TACOM  is 
acting  as  DARPA’s  ag.ent.  Some  particulars  of  this  project  are: 

-  June  1991  to  June  1994  period  of  performance. 
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-  Objective  is  to  demonstrate  real-time  passive  3-D  sensing  and  navigation  in 

u  '.structured  domains.  “An  autonomous,  outdoor,  mobile  UGV  system 
will  be  developed  capuole  of  accomplishing  real-time  mobile  robot 
navigation/driving  in  a  variety  of  non-tnvial/natural  outdoor  domains  with 
increasing  levels  of  autonomous  capability”  (SOW  of  contract). 

Image  Understanding  Architecture  Prototype  Evaluation  and  Development^  in  conjunction 
with  Hughes  Research  Lab  [hardware]  and  Amerinex  Artificial  Intelligence,  Inc. [software]) 
is  being  contracted  through  Harry  Diamond  Lab.  Some  particulars  of  this  project  are: 

-  Objective  is  to  build  one  prototype  of  the  Image  Understanding  Architecture 

for  evaluation  of  3  level  paralle'  processing  concepts. 

Figure  14  illustrates  graphically  what  functionalities  UMass  Amherst  is  working  on  in 
relation  to  the  required  technologies  list  in  Figure  7. 
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Figure  14.  UMass  Department  of  Computer  Science  Contributions  to 
Required  Technologies  Checklist  Derived  From  Figure  7 


7.7  ROBOTIC  SYSTEMS  TECHNOLOGY,  INC. 

RST  is  developing  the  Surrogate  Teleoperated  Vehicle  (STV)  for  the  Tactical  Unmanned 
Ground  Vehicle  Joint  Program  Office  (TUGV-JPO).  The  short  term  objective  of  the 
TUGV-JPO  is  to  provide  a  mobility  platform  capable  of  transporting  a  RSTA  package. 
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Part  of  the  RSTA  hardware  package  is  also  being  developed  under  this  project.  The  first 
delivery  is  programmed  for  the  first  quarter  of  FY  1992. 


Some  features  of  the  STV  project  include: 

•  Use  of  a  commercially  available  mobility  platform 

•  Modular  RSTA  package,  including  a  laser  designator  and  a  FLIR. 

•  DC  Electric  drive  option  of  quiet  operation. 

•  Standardization  of  architecture  (VMEbus  and  VxWorks  development  environment 
for  teleoperated  TUGV's). 

•  A  GPS  receiver  for  navigation. 

•  An  acoustic  detection  system  with  a  limited  automatic  target  cueing  capability. 

•  A  combined  fibre  optic  and  radio  communication  system. 

•  A  man-portable  OCU. 

Figure  15  illustrates  graphically  what  functionalities  RST  is  working  on  in  relation  to  the 
required  technologies  list  in  Figure  7. 
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Figure  15.  RST  Contributions  to  Required  Functionalities  Checklist 

Derived  From  Figure  7 
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7 . 8  DEPARTMENT  OF  ENERGY 

The  Department  of  Energy  (DOE)  research  and  development  activity  in  robotics  is 
focused  at  Oak  Ridge  and  Sandia  National  Laboratories. 

Qak.Ri.dse 

The  Oak  Ridge  program,  consists  of  two  major  research  areas: 

1)  robotics  and  teleoperations  (focus  on  telepresence) 

2)  machine  intelligence  and  advanced  computer  systems  group. 

The  robotics  and  teleoperations  area  includes  groups  working  in  teleoperation  and  remote 
systems,  robotic  sensors  and  electronics  development  and  robotic  mobility  and 
manipulation.  Much  of  this  work  focuses  on  teleoperations  for  Nuclear  Fuel  reprocessing 
and  is  in  the  area  of  mechanisms,  dynamics  and  controls  for  dexterous  manipulation. 
These  aspects  of  the  research  are  of  limited  interest  to  autonomous  navigation. 

However,  the  sensors  and  vision  aspects  of  the  work  and  three-dimensional  world  model 
representation  and  update  have  direct  analogy  to  the  needs  for  three-dimensional  space 
understanding  for  autonomous  navigation.  The  machine  intelligence  and  advanced 
computer  systems  area  includes  groups  working  on  planning,  reasoning  and  problem 
solving.  The  work  on  mobile  robots,  while  limited  to  constrained  environments  includes 
several  aspects  applicable  to  militaty  autonomous  navigation  problems. 

The  activities  supported  by  this  group  includes  world  modeling  at  the  University  of 
Florida,  obstacle  avoidance  using  sonar  and  navigation  using  stereo  visible  imagery  at  the 
University  of  Michigan,  manipulation  using  tactile  and  proximity  sensors  and  system 
integration  at  the  University  of  Tennessee  and  manipulation  at  the  University  of  Texas. 
The  implementation  of  this  work  on  the  mobile  robot  at  Oak  Ridge,  while  confined  to  an 
indoor  environment,  provides  some  of  the  fundamental  technology  applicable  to  the 
military  unmanned  ground  vehicle  navigation  problem.  They  are  also  working  in  a 
collaborative  effort  with  DOD  in  the  development  of  data  compression  algorithms  for 
DEMO  I  and  integrating  an  in-house  composite  compression  algorithm  with  MITs 
prediction  image  compression  algorithm. 

Figure  16  illustrates  Oak  Ridge  National  Laboratories  contributions  to  the  UGV  checklist 
developed  in  Figure  7. 
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Figure  16.  Oak  Ridge  National  Laboratory’s  Contributions  to  Required 
Functionalities  Checklist  Derived  From  Figure  7 

Sandia 

Sandia  has  been  previously  involved  in  two  robotic  vehicle  programs  (TMAP  and  RSV) 
and  is  currently  involved  in  a  third  program  (RETRVTR). 


TMAP  produced  two  brassboard  vehicles,  manufactured  by  Grumman  and  Martin.  The 
contact  at  Sandia  is  Mr.  Bill  Caskey,  project  leader,  (505)  844-8835.  This  activity  was 
funded  out  of  MICOM.  Products  were  transferred  from  Sandia  to  the  UGV  JPO  when  that 
office  was  formed.  The  TMAP  program  is  complete.  Sandia  involvement  in  this  program 
ended  when  TMAP  finished.  The  Martin  vehicle  had  significantly  better  overall 
performance  and  JPO  has  repons  documenting  the  vehicles  and  the  test  regimens.  Martin 
delivered  their  vehicle  to  JPO  (Huntsville)  where  it  remains  essentially  unused  except,  for 
infrequent  VIP  demos  of  the  technology.  The  Grumman  vehicle  was  never  delivered  and,at 
Grumman’s  request,  they  retained  possession  as  part  of  an  IR&D  effort  to  improve  and 
market  the  capability  internationally.  From  a  technical  perspective,  both  vehicles  were 
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teleoperated.  Navigation  was  provided  by  the  telcoperator  using  live  TV  imagery' 
transmitted  back  to  the  remote  control  site  over  a  fiber  link  (RF  was  available  as  backup). 
Several  ergonomic  experiments  were  conducted  in  which  the  reliability  of  the  teleoperator 
to  navigate  were  tested.  These  experiments  involved  steerable  cameras,  color  vs.  B&W, 
and  variable  bandwidth  of  the  communications  channel.  Each  platform  did  have  a  dead 
reckoning  capability  to  provide  the  operator  with  a  coarse  approximation  of  map  location. 
The  computed  map  location  was  updated  by  the  operator  based  on  visual  recognition  of 
landmarks  in  the  transmitted  imagery.  The  systems  had  no  proximity  sensing  capabilities. 

Robotic  Security,  Vehicle  (Program  complete) 

Under  contract  from  the  Defense  Nuclear  Agency,  Sandia  has  developed  and  tested  the 
Robotic  Security  Vehicle  (RSV),  a  brassboard  system  to  perform  automatic  navigation  of  a 
land  vehicle  in  a  structured  (i.e.,  previously  mapped)  environment  The  Sandia  contact  is 
Paul  R.  Klarer,  project  engineer,  (505)  846-2974.  The  vehicle  used  was  the  Jeep 
Cherokee  Mobile  Robotics  Testbed  Vehicle,  developed  at  Sandia.  The  target  application  for 
this  project  is  outdoor  site  security  applications.  The  target  market  is  storage  and  warehouse 
DOD  sites  which  have  remote  sentry  requirements.  The  status  of  the  program  is  operational 
-  a  capability  has  been  demonstrated  with  the  Jeep.  Sandia  is  planning  to  port  the 
capabilities  to  smaller,  more  cost  effective  vehicles,  more  suitable  to  the  target  application. 

The  RSV  system  uses  on-board  dead  reckoning  navigation,  derived  from  a  fluxgate  and  an 
odometer,  to  determine  position  and  orientation  of  the  vehicle  in  real  time.  That  data  is 
combined  with  periodic  updates  from  an  external  microwave  beacon  system  to  reduce 
position  errors  to  within  reasonable  limits.  The  roads  or  trails  that  the  system  operates  on 
are  stored  on-board  as  maps,  and  these  maps  define  the  navigation  system's  limit  of 
knowledge  of  its  surroundings.  The  use  of  an  a  priori  map  decreases  the  systems  utility  for 
autonomous  exploration  in  unknown  territory,  but  has  other  advantages  in  terms  of  reduced 
computing,  lower  power  and  size  requirements.  Additionally,  the  need  for  favorable 
lighting  is  eliminated,  except  where  vision  is  used  as  an  adjunct  to  the  navigation  system.  A 
local  obstacle  avoidance  capability  is  included  and  is  handled  by  a  separate  sensor  system, 
using  simple  sonic  range  detection  sensors.  This  results  in  real  time  navigation  capabilities 
which  are  enhanced  in  terms  of  vehicle  speed  and  continuous  motion  capability.  A  present 
disadvantage  of  the  system  is  the  inability  of  the  local  obstacle  avoidance  system  to  detect 
potholes  and  the  inability  to  automatically  adjust  driving  velocity. 

RETRVIR  (Program  ongoing) 

The  program  currently  in  progress  is  the  Remote  Telerobotic  Vehicle  for  Intelligent 
Remediation  (RETRVIR).  This  is  a  Sandia  IR&D  program  to  integrate  a  sensor-based  high 
dexterity  manipulator  with  a  remote-controlled  robotic  vehicle  for  characterization  and 
cleanup  of  hazardous  waste  sites.  The  Sandia  contacts  are  Ray  Harrigan,  Manager  of  the 
Intelligent  Machine  Division,  (505)  844-3004  and  Chris  Wilson,  project  engineer,  (505) 
844-5276.  This  program  uses  a  Honda  vehicle  which  has  been  modified  for  remote 
driving  control.  It  is  a  teleoperated  system  and,  as  such,  does  not  yet  have  any  autonomous 
navigation  or  local  obstacle  avoidance  capabilities.  Navigation  is  provided  by  the  tele¬ 
operator  using  live  TV  imagery  transmitted  back  to  the  remote  control  site  over  a  fiber  link 
(RF  is  available  as  backup).  The  system  is  operational  and  the  focus  for  near  term  work  is 
to  integrate  advanced  sensors  with  the  manipulator  arm  for  improving  controlled 
interactions  with  the  environment.  Force  sensing  at  the  end  effector  is  an  area  of  research  in 
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this  project.  Future  plans  call  for  integration  of  a  local  obstacle  avoidance  system  using 
sonic  sensors  and  a  reflexive/  potential  field  algorithm  for  real-time  obstacle  avoidance. 

Figure  17  illustrates  graphically  the  areas  that  DOE  is  working  that  are  applicable  to  UGV 
areas  or  technology. 
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Figure  17.  Sandia  National  Laboratory’s  Contributions  to  Required 
Functionalities  Checklist  Derived  From  Figure  7 


7 . 9  NAVAL  OCEAN  SYSTEMS  CENTER 
7.9.1  Autonomous  Sentry 

Program  sponsored  by  the  Army  to  assist  DoD  in  developing  operational  indoor  sentry 
systems.  A  secondary  objective  might  be  to  assist  the  Department  of  Energy  in  developing 
robots  for  monitoring  toxic  waste  containers.  NOSC  had  previously  addressed  the 
problems  of  an  autonomous  sentry  in  an  internal  project  that  was  viewed  as  an  exercise  in 
developing  expertise  in  robotics  systems. 

A  small  sentry  robot  is  equipped  with  multiple  sensors  to  detect  sound,  motion,  light  and 
other  observables.  Three  alternative  schemes  for  navigation  are  being  pursued: 

1)  Guide  path 

2)  Unrestricted  path  planning  on  a  two-dimensional  grid 

3)  Cybermotion  virtual  path  technique. 
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Navigation  planning  is  accomplished  off-line  on  a  PC  and  final  instructions  radioed  to  the 
robot. 

Prototypes  have  existed  since  1988  and  demonstrations  will  continue  for  the  next  few 
years.  Continued  research  will  include  implementing  technique  on  different  platforms. 

7.9.2  Teleoperated  Vehicle 

Demonstration  of  teleoperation  of  up  to  three  vehicles,  exercises  to  include  live  firing. 
Stripped  down  HMMWV’s  were  used  to  demonstrate  reconnaissance,  surveillance,  and 
target  acquisition  (RSTA)  and  munitions  firing.  One  vehicle  was  used  only  for  RSTA,  one 
vehicle  was  used  only  for  firing  and  one  was  used  for  both.  Fiber  optic  cables  used  to  pass 
data  to  remote  operator  in  OCU  with  controls  like  HMMWV. 

Successful  demonstration  at  Camp  Pendleton  in  September,  1989.  Nine  missiles  fired  with 
nine  bullseyes  and  vehicle  achieved  25  MPH  speed. 

The  products  of  this  program  were  transferred  to  the  UGV  JPO  under  LCOL  Harper. 


7.10  NASA  AND  RELATED  RESEARCH 
7. 10. 1  Overview  of  NASA's  Telerobotics  Program 

NASA  initiated  the  Telerobotics  Research  Program  in  1985.  This  program  was  intended  to 
foster  the  development  of  telerobotics  technology  for  eventual  application  in  the  space 
program.  According  to  Weisbin  and  Montemerlo,  NASA's  telerobotics  program  can  be 
divided  into  three  areas: 

•  Teleoperation  -  a  human  directly  controls  the  remote  device  in  real  time, 

•  Robotics  -  the  remote  device  is  preprogrammed,  and 

•  Supervisory  Control  -  the  human  controller  gives  high  level  commands  which  are 
decomposed  and  executed  by  the  machine  under  human  supervision. 

In  the  first  three  years  of  the  telerobotics  program,  an  emphasis  was  placed  on  developing 
generalized  telerobotics  capabilities;  specific  tasks  and  applications  were  not  drivers  of  the 
research.  In  the  last  four  years,  however,  the  emphasis  has  changed.  Smaller  programs 
with  specific  requirements  were  created,  thus  allowing  rapid  prototyping  of  systems  so  that 
the  strengths  and  weaknesses  of  a  given  approach  could  be  determined  quickly.  Typically, 
the  time  from  concept  development  to  deployment  in  an  actual  application  can  be  one  to  two 
decades.  At  this  point,  the  Shuttle  Remote  Manipulator  is  the  only  robot  with  real  space 
experience. 

Funding  for  the  Telerobotics  Program  and  related  activities  has  come  primarily  through  the 
Office  of  Aeronautics  and  Space  Technology  (OAST),  which  is  now  known  as  the  Office 
of  Aeronautics  Exploration  and  Technology  (OAET),  although  other  offices  have  funded 
related  research  programs  as  well.  University  grants  are  also  used  to  promote  research  in 
key  component  areas.  Examples  include  major  universities  such  as  Stanford,  Carnegie 
Mellon,  Maryland,  and  Texas.  The  Office  of  Commercial  Programs  at  NASA  is  increasing 
industry  participation  in  automation  and  robotics  through  its  Centers  for  the  Commercial 
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Development  of  Space  (CCDS).  These  centers  are  funded  typically  through  $1  M  base 
grants  that  is  used  as  seed  funding  for  joint  programs  with  industry.  As  an  example,  the 
Space  Automation  and  Robotics  Center  (SpARC)  at  the  Environmental  Research  Institute 
of  Michigan  is  involved  in  autonomous,  supervised,  guided,  and  teleoperated  robotics. 

The  CCDS's  operate  differently  from  other  programs  in  that  available  (current)  technology 
is  utilized  to  a  greater  extent  and  industry  participation  (both  cash  and  in-kind)  is  essential. 
These  programs  are  by  definition  lower  cost  and  the  turn  around  time  from  concept  to 
delivery  is  generally  on  the  order  of  several  years,  rather  than  decades. 

The  most  relevant  NASA  program  to  UGVs  is  the  MARS  land  rover,  which  will  be 
discussed  below.  This  program  has  pote  ial  for  contributing  substantial  technology 
research  to  the  UGV  program. 

7. 10.2  Representative  Listing  of  NASA's  Tele: obotics/Rover  Activities 

The  first  pan  of  this  section  addresses  NASA's  programs  to  develop  rovers  for  future 
Lunar  and  Mars  missions.  The  second  part  contains  a  list  of  selected  programs  read  from 
the  literature.  Research  is  highlighted  that  is  going  on  in  space  and  terrestrial  telerobotics. 

•  Lunar! Mars  Rover  Programs 

The  primary  effort  to  create  autonomous  robots,  or  rovers  for  Lunar  and  Mars  missions  is 
taking  place  at  NASA's  Jet  Propulsion  Lab  (JPL)  and  Carnegie  Mellon  University.  Both 
unmanned  and  piloted  rovers  are  being  developed.  The  unmanned  rovers  are  intended  for 
survey,  exploration,  and  sampling  missions  and  the  piloted  rovers  are  needed  for  long 
range  transportation  on  planetary  surfaces. 

Two  types  of  rovers  are  currently  under  investigation,  with  prototypes  of  each  already 
being  tested.  At  Carnegie  Mellon's  Robotics  Institute,  researchers  have  built  a  six-legged, 
19  foot  tall  walking  machine  called  the  Ambler  (Autonomous  Mobile  Exploration  Robot). 
Work  on  the  Ambler  has  progressed  since  1987,  with  its  first  steps  being  taken  in  the 
laboratory  in  1990.  The  Ambler  uses  a  laser  scanner  to  provide  range  images  for  path 
planning  and  a  Sun  computer  to  provide  the  processing  power.  The  Ambler  is  capable  of 
stepping  over  obstacles  1  m  high  and  maintaining  a  stable,  level  platform  on  30  degree 
slopes. 

At  JPL,  researchers  have  taken  a  different  approach  to  rover  design.  A  semi-autonomous, 
wheeled  rover  prototype  called  Robby  has  been  developed  and  is  being  tested  in  outdoor 
terrain.  Robby  is  a  six-wheeled,  three-body  rover .  It  is  approximately  13  feet  long,  5  feet 
wide,  6.5  feet  high,  and  weighs  2500  lbs.  With  35  inch  diameter  wheels,  Robby  can  also 
climb  over  1  m  high  obstacles.  Robby  also  has  a  commercial  robot  arm  on  its  front  section 
for  sample  collection  purposes.  The  rover  has  two  separate  processors:  one  for  perception 
and  planning,  and  the  other  for  wheel  and  robot  arm  control.  Robby  made  its  first 
continuous,  semi -autonomous  journey  over  rough  terrain  in  May  of  1990.  A  stereo  vision 
system  provided  input  for  Owk/kuv!c  avoids  r.ce  to  supplement  a  general,  pre-planned  route. 
JPL  researchers  are  also  developing  a  computer-aided  remote  driving  (CARD)  capability 
for  Robby  that  allows  more  human  control  over  the  robot.  JPL  has  also  developed  a  test 
vehicle  for  the  U.S.  Army  Tank- Automotive  Command  that  takes  advantage  of  the  CARD 
technology.  The  army  test  vehicle  has  a  van  for  remote  command  and  control.  An  operator 
in  the  van  can  communicate  with  the  test  vehicle  over  a  reduced  bandwidth  communication 
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channel.  Stereo  images  are  transmitted  to  the  operator,  who  can  then  plan  a  safe  path  for 
the  vehicle. 

•Space  Related 

Structural  Assembly 


Langley  Research  Center  (LaRC)  -  in  orbit  telerobotic  assembly  of  large  space  vehicles  and 
platforms 

Supervisory. Control  of  Telerobotics 


LaRC  -  in  support  of  truss  construction,  automated  housekeeping  for  Space  Station 
Freedom  Lab  Module,  and  telerobotic  maintenance  of  external  modules 

Supervised  Inspection 

JPL  -  routine  and  unscheduled  inspection  tasks,  such  as  truss  struts 
Advanced  Teleoperation 

JPL  -  demonstrating/evaluating  capabilities  for  space  repair  tasks 

Exoskeleton  Anthropomorphic  Telemanipulation 

JPL  -  man-equivalent  dexterity  of  remotely  operated  hands 


Failure  Tolerant  Manipulator  Joint  Development 

JSC,  University  of  Texas,  Austin  -  fault  tolerant  architectures 

*  University  Research 


Stanford  University  (ARL)  -  precise  manipulation  of  objects  in  free  space 

Neutral. Buoyancy  Research 


University  of  Maryland  Space  Systems  Lab  (SSL)  -  neutral  buoyancy  simulations  of 
EVA/telerobotic  work  sites 

•  Terrestrial  Robotics 

Emergency  Response  Robot 

JPL  -  for  treating  hazardous  spill  releases 

Vacuum  Plasma  Sprav  Robot 


Marshall  Space  Flight  Center  -  automating  VPS  for  fabricating  space  transportation  system 
engine  hardware 

Tile, .Inspection  and  ,.Re.wat£rproofmg 

Kennedy  Space  Center,  Carnegie  Mellon,  Stanford  Research  Institute,  Langley  Research 
Institute  -  ground  processing  of  space  constructions 

Aerial  Inspection  Robot 

Kennedy  Space  Center  -  automatic  inspection  of  filters  for  payload  changeout  room. 
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Figure  18  illustrates  graphically  the  functionalities  JPL  is  working  against  those  identified 
in  Figure  7. 


Levels  or  Autonomy 
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Figure  18.  Jet  Propulsion  Laboratory's  Contributions  to  Required 
Functionalities  Checklist  Derived  From  Figure  7 


7.11  NATIONAL  INSTITUTE  OF  STANDARDS  AND  TECHNOLOGY 

NIST  is  involved  in  two  different  technical  thrusts,  control  architectures  and  optical  flow. 
The  principal  point  of  contact  was  Mr.  Richard  Quintero.  The  major  effort  is  in  the 
definition  of  control  architectures  for  intelligent  systems,  an  effort  pioneered  by  James 
Albus  of  NIST. 

Another  technical  effort  under  development  using  optical  flow  for  depth  understanding. 

Dr.  Tsai  Hong  is  the  principal  point  of  contact  on  this  effort. 

NIST  is  part  of  the  DEMO  I  effort  and  is  providing  the  expertise  in  the  system  integration 
area  for  the  effort  as  well  as  installing  the  low  level  actuator  package  for  one  of  the  DEMO  I 
vehicles. 
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In  terms  of  defining  an  architecture,  NIST  has  defined  the  following  elements  of  an 
intelligent  system. 

a.  Actuators 

b.  Sensors 

c.  Sensory  processing 

d.  World  Model 

e.  Values 

f.  Task  decomposition 

These  elements  are  combined  in  an  hierarchical  fashion  in  order  to  reduce  the  amount  of 
information  at  higher  levels  of  abstraction  and  reasoning.  This  view  of  an  architecture 
requires  that  global  memory  can  be  accessed  by  all  processes  and  that  scalable,  parallel 
computing  is  available.  This  architecture  more  closely  follows  the  single-instruction- 
multiple-data  (SIMD)  paradigm  for  parallel  processing. 

Figure  19  illustrates  NIST’s  vision  of  intelligent  control  system  that  they  call  Robotic 
Control  System  (RCS). 


Figure  19.  The  Elements  of  Intelligence  and  the  Functional  Relationships 

Between  Them 

The  image  flow  work  is  based  on  the  primary  assumption  of  small  differences  in  time  (high 
frame  rate)  which  will  allow  tracking  of  individual  pixels  as  they  move  in  space.  It  was 
found  that  this  kind  of  algorithm,  being  a  brute  force  method,  requires  tremendous 
computing  power.  The  current  problem  is  in  mapping  the  algorithm. 
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The  system  architecture  work  is  crucially  important  to  adequate  development  of  unmanned 
ground  vehicles.  The  image  flow  work,  though  interesting,  probably  won’t  have  as  much 
impact  in  the  current  study  as  understanding  the  architecture  work.  Figure  20  illustrates 
graphically  the  functionalities  that  the  National  Institute  of  Standards  and  Technology  is 
working  against  those  identified  in  Figure  7. 


Figure  20.  National  Institute  of  Standards  and  Technology’s  Contributions 
to  Required  Functionalities  Checklist  Derived  From  Figure  7 


7.12  HUMAN  ENGINEERING  LABORATORY 

The  principal  point  of  contact  was  Mr.  Tom  Haduch.  The  facilities  the  Human  Engineering 
Laboratory  operates,  includes  an  indoor  laboratory  for  human  factors  testing  related  to 
remotely  operating  a  small  vehicle. 

Currently,  HEL  is  pursuing  three  distinct  efforts,  two  as  contractual  monitors,  and  one 
effort  as  a  research  center  in  its  own  right.  They  are: 

•  Man-machine  interface  research  (on-site  at  Human  Engineering  Laboratory). 

•  Low  data  rate  communication  research  (contract  to  Oak  Ridge  National  Laboratory). 

•  High  level  control  architecture  (contract  to  National  Institute  of  Standard  and  Technology). 
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Within  MMI  research,  they  have  identified  five  distinct  areas  for  further  research.  They 
are: 

1)  Controls. 

2)  Vision. 

3)  Information. 

4)  Response. 

5)  Display  design. 

A  further  area  of  research  not  so  easily  categorized  is  communication  or  display  image 
refresh  rate,  which  is  affected  by  the  demands  in  the  five  areas  above. 

According  to  Haduch,  MMI  and  human  factors  in  general  are  at  an  immature  stage  of 
development.  There  is  still  significant  effort  in  MMI  research  needed  in  order  to  achieve 
adequate  teleoperation.  Some  of  this  will  be  provided  from  Demo  I  and  the  STV 
evaluations.  The  state  of  development  of  MMI  should  be  reviewed  after  these  activities  are 
completed. 

Figure  21  illustrates  the  technologies  that  are  being  researched  directly  at  the  Human 
Engineering  Laboratory  (HEL),  excluding  work  for  which  HEL  is  acting  as  contract 
monitor . 
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Navigation  Sensors 
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Environment  Model 


Sensor  Preprocessing 


Computer  'Vision  Architectures 


High  Bandwidth  Communication 


Low  Bandwidth  Comrr  anication 


Figure  21.  U.  S,  Army  Human  Engineering  Laboratory’s  Contributions  to 
Required  Functionalities  Checklist  Derived  From  Figure  7 

7.13  UNIVERSITY  OF  MARYLAND 

The  principal  efforts  at  the  University  of  Maryland  include  the  exploration  of  the  massively 
parallel  Connection  Machine  for  mission  planning  algorithms.  One  of  the  particular  areas 
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of  research  was  in  developing  methods  to  direct  a  group  of  vehicles  to  act  cooperatively, 
using  algorithms  developed  on  the  Connection  Machine.  In  the  past,  the  University  of 
Maryland  had  contributed  to  the  Autonomous  Land  Vehicle  program  by  providing  advice  to 
Martin  Marietta  on  road  following  algorithms. 

Also  of  interest  is  some  exploratory  work  by  Aloin  Monos  of  the  University  of  Maryland. 
Dr.  Monos  is  pursuing  visual  navigation  without  resort  to  a  three  dimensional  environment 
model,  which,  if  successful,  would  reduce  requirements  on  sensors;  an  interesting,  if 
early,  research  trend.  Figure  22  illustrates  the  contributions  of  the  University  of  Maryland 
towards  the  required  functionalities  in  Figure  7. 


Figure  22.  University  of  Maryland’s  Contributions  to  Required 
Functionalities  Checklist  Derived  From  Figure  7 


7.14  UNIVERSITY  OF  MICHIGAN 

The  University  of  Michigan  is  contributing  in  the  areas  of  cognitive  reasoning,  perception, 
world  and  environment  modeling,  obstacle  avoidance  and  multiple  agent  planning.  The 
principal  point  of  contact  is  Prof.  Ramesh  Jain.  Some  of  the  more  exploratory  work  that 
they  are  pursuing  is  in  how  to  fu  ;e  sensor  data  from  multiple  sources.  Prof.  Jain  argues 
that  fusion  should  occur  through  assimilation,  that  is,  in  the  building  of  the  environment 
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model  »n  the  cognition  process  (refer  to  Figure  6).  In  this  way,  uncertainty  introduced  by 
each  sensor  can  be  accounted  for  when  updating  the  environment  model. 

The  University  of  Michigan  is  also  contributing  to  DEMO  II  in  the  area  of  multiple  agent 
planning.  Dr.  Terry  Weymouth  is  the  principal  point  of  contact  in  the  DEMO  II  effort.  In 
order  to  reduce  communication  bandwidth  and  deal  constructively  with  uncertainty,  U  of 
Mich  has  developed  a  technique  called  Partial  Global  Planning  (PGP).  In  this 
methodology,  local  detailed  plans  are  form  dated  for  each  agent,  based  on  local  conditions 
and  the  role  the  agent  is  supposed  to  play.  These  local  plans  are  then  abstracted  and 
communicated  to  the  other  agents  in  the  formation.  In  this  way,  communication  bandwidth 
is  reduced,  commitment  to  detail  is  avoided  and  meaningless  detail  is  not  transmitted. 

Figure  23  illustrates  the  University  of  Michigan’s  contributions  towards  the  functionalities 
in  Figure  7. 
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Figure  23.  University  of  Michigan’s  Contributions  to  Required 
Functionalities  Checklist  Derived  From  Figure  7 
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7.15  CARNEGIE  MELLON  UNIVERSITY 

The  group  at  Carnegie  Mellon  has  been  very  active  in  several  aspects  of  research  and 
demonstration  development  in  autonomous  navigation  systems.  In  particular  they  have 
been  involved  in  sensor  system  utilization  and  evaluation  for  autonomous  navigation,  road 
following  algorithm  development  and  demonstration  for  autonomous  navigation,  the 
construction  and  generation  of  NAVLAB  (an  autonomous  navigation  demonstration 
system),  and  the  development  and  utilization  of  iWarp  for  autonomous  navigation  image 
processing. 

Active  sensor  systems  for  autonomous  navigation  evaluated  at  Carnegie  Mellon  include 
active  3-D  imaging  systems  built  by  ERIM,  Odetics  and  Perceptron.  Sensor  systems 
utilized  for  autonomous  navigation  include  3-D  range  imaging,  color  video,  FLIR,  INS, 
and  range  scanners,  demonstrations  of  road  following  using  these  systems  on  NAVLAB 
and  other  vehicles  have  been  developed. 

Road  following  algorithm  development  has  focused  on  the  development  and  utilization  of 
ALVINN  (Autonomous  Land  Vehicle  In  a  Neural  Network).  This  is  a  3-layer  back- 
propagation  network  designed  for  the  task  of  road  following.  Currently  ALVINN  takes 
images  from  a  camera  and  a  laser  range  finder  as  input  and  produces  as  output  the  direction 
the  vehicle  should  travel  in  order  to  follow  the  road.  ALVINN  is  a  connectionist  approach 
to  the  navigational  task  of  road  following.  Specifically,  ALVINN  is  an  artificial  neural 
network  designed  to  control  the  NAVLAB,  the  Carnegie  Mellon  autonomous  navigation 
test  vehicle. 

ALVINN's  current  architecture  consists  of  a  single  hidden  layer  back-propagation  network. 
The  input  layer  is  divided  into  three  sets  of  units:  two  "retinas"  and  a  single  intensity 
feedback  unit.  The  two  retinas  correspond  to  the  two  forms  of  sensory  input  available  on 
the  NAVLAB  vehicle;  video  and  range  information.  The  first  retina,  consisting  of  30x32 
units,  receives  video  camera  input  from  a  road  scene.  The  activation  level  of  each  unit  in 
this  retina  is  proportional  to  the  intensity  in  the  blue  color  band  of  the  corresponding  patch 
of  the  image.  The  blue  band  of  the  color  image  is  used  because  it  provides  the  highest 
contrast  between  the  road  and  the  non-road.  The  second  retina,  consisting  of  8x32  units, 
receives  input  from  a  laser  range  finder.  The  activation  level  of  each  unit  in  this  retina  is 
proportional  to  the  proximity  of  the  corresponding  area  in  the  image.  The  road  intensity 
feedback  unit  indicates  whether  the  road  is  lighter  or  darker  than  the  non-road  in  the 
previous  image.  Each  of  these  1217  input  units  is  fully  connected  to  the  hidden  layer  of  29 
units,  which  is  in  turn  fully  connected  to  the  output  layer. 

The  output  layer  consists  of  46  units,  divided  into  two  groups.  The  first  set  of  45  units  is  a 
linear  representation  of  the  turn  curvature  along  which  the  vehicle  should  travel  in  order  to 
head  towards  the  road  center.  The  middle  unit  represents  the  "travel  straight  ahead" 
condition  while  units  to  the  left  and  right  of  the  center  represent  successively  sharper  left 
and  right  turns.  The  network  is  trained  with  a  desired  output  vector  of  all  zeros  except  for  a 
"hill"  of  activation  centered  on  the  unit  representing  the  correct  turn  curvature,  which  is  the 
curvature  which  would  bring  the  vehicle  to  the  road  center  7  meters  ahead  of  its  current 
position.  More  specifically,  the  desired  activation  levels  for  the  nine  units  centered  around 
the  correct  turn  curvature  unit  are  0.10,  0.32,  0.61,  0.89,  1.00,  0.89,  0.61, 0.32,  and 
0.10.  During  testing,  the  turn  curvature  dictated  by  the  network  is  taken  to  be  the  curvature 
represented  by  the  output  unit  with  the  highest  activation  level.  The  final  output  unit  is  a 
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road  intensify  feedback  unit  which  indicates  whether  the  road  is  lighter  or  darker  than  the 
non-road  in  the  current  image. 

iWarp  is  the  image  processor  which  has  been  developed  by  Intel  and  utilized  at  Carnegie 
Mellon  for  autonomous  navigation  image  processing.  iWarp  is  a  programmable  systolic 
array  processor  of  10  cells,  each  of  which  is  a  processor  capable  of  performing  10  million 
floating-point  operations  per  second  (10  MFLOPS).  A  10-cell  machine,  therefore,  has  a 
peak  performance  of  100  MFLOPS.  The  Warp  machine  has  three  major  components  —  the 
Warp  array  (10  cells),  an  interface  unit  (IU),  and  a  host  system  consisting  of  three  68020 
processors  and  a  Sun  Microsystems  workstation.  The  Warp  array  performs  the 
computation-intensive  operations.  The  IU  handles  the  I/O  between  the  array  and  the  host. 
The  host  executes  the  parts  of  the  application  program  that  are  not  mapped  onto  the  Warp 
array  and  supplies  the  data  to  and  receives  results  from  the  array.  The  Warp  machine  is 
programmed  in  a  special  purpose  high  level  language  called  W2.  The  language  W2  is 
supported  by  an  optimizing  compiler  that  translates  W2  programs  into  efficient  microcode 
for  the  Warp  cells  and  the  IU,  and  C  code  for  the  host  processors.  The  NAVLAB  I  vehicle 
which  incorporated  these  system  has  been  demonstrated  doing  road  following  and  obstacle 
avoidance  on  clearly  defined  paved  roads  at  speeds  of  normal  traffic.  Work  is  continuing 
on  the  evolution  of  this  system  to  make  it  more  robust  and  reliable. 

In  addition,  Carnegie  Mellon  has  the  NAVLAB  II  vehicle  for  demonstration  on  gravel  and 
dirt  roads.  NAVLAB  II  is  a  modified  HMMWV  ambulance  currently  equipped  with  an 
ERIM  range  scanner,  video  camera,  and  an  inertial  navigation  system.  In  the  future,  a 
stabilized  sensor  platform  will  be  mounted  on  top  of  the  vehicle,  housing  a  Schwartz  single 
scanline  laser  sensor  to  replace  EREM's  (for  faster  update  rate),  a  FLIR,  GPS,  a  video 
camera,  and  an  INS  VNAS.  The  computing  system  consists  of  three  Sun  Sparc2 
workstations,  4-7  image  buffers,  1.6  GB  disk  storage,  an  optical  disk,  Vx  Works  real-time 
operating  system  software,  and  4-axis  motion  controllers.  This  equipment  is  powered  by 
two  gas-powered  generators  with  a  capacity  of  10  KW.  In  addition,  there  are  four  seats, 
three  for  researchers  and  one  for  an  emergency  driver. 

CMU  is  initiating  some  new  research  in  the  area  of  high  speed  autonomous  navigation  on 
rough  terrain.  CMU  is  currently  building  a  base-line  system  to  operate  at  moderate  speeds 
and  plans  to  incrementally  increase  the  high-end  operating  speeds  of  the  vehicle.  The  goal 
is  to  perform  velocity  tuning  so  the  vehicle  drives  at  an  appropriate  speed  for  the 
complexity  of  the  terrain  and  the  difficulty  of  the  planned  path.  A  local  path  planner  first 
plots  a  course  in  spatial  terms,  avoiding  obstacles  while  meeting  global  path  needs.  It  then 
formulates  an  appropriate  velocity  plan  meeting  the  kinematic  and  dynamic  constraints  of 
the  vehicle  along  the  planned  path.  This  system  currently  operates  in  simulation  mode  and 
will  be  transported  to  NAVLAB  II. 
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Figure  24.  Carnegie  Mellon  University’s  Contributions  to  the 
Functionalities  Described  in  Figure  7 


7.16  U.  S.  ARMY’S  TANK  AUTOMOTIVE  COMMAND 

Prior  to  the  Army’s  shift  of  UGV  related  development  program  responsibility  to  Micom 
(FY90),  TACOM  was  one  of  the  key  centers  of  UGV  research  and  development  It  is  still 
a  major  participant  in  DEMO  I  and  continues  to  serve  as  a  contracting  agent  and  Army  focal 
point  for  DEMO  13  in  technical  areas  of  planning  and  navigation.  TACOM  has  a  mix  of 
goals  and  objectives,  ranging  from  developing  fieldable  teleoperated  systems  for 
deployment  in  Desert  Storm  to  contributing  managerially  and  technically  to  the  development 
of  autonomous  and  cooperative  unmanned  vehicles. 

TACOM  is  active  in  UGV  robotics  development  from  a  total  system  approach,  elements  of 
which  include  navigation,  perception,  mobility,  communication,  solder-machine  interface 
and  mission  function  automation.  Demo's  I  and  II  drive  the  TACOM  plan  for  Tank- 
Automotive  Robotics.  Other  projects  of  interest  of  historical  interest  include  the  Unmanned 
Ground  Vehicle  Control  Testbed  (UGVCT)  and  the  LABCOM  Robotically  Controlled  High 
Mobility  Multipurpose  Wheeled  Vehicle. 

The  Unmanned  Ground  Vehicle  Control  Testbed,  developed  by  the  U.S.  Army  Tank- 
Automotive  Command,  provides  a  means  for  evaluating  how  robotic  vehicle  units  can 
accomplish  combat  and  combat  support  missions.  Using  digital  terrain  data  base  products. 
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automated  destination  selection  and  route  planning,  and  computer  assisted  driving,  the 
testbed  can  remotely  control  up  to  four  robotic  vehicles  simultaneously  and  can  demonstrate 
six  mission  capabilities: 

•  Decoy  operation 

•  Tactical  reconnaissance 

•  Chemical  reconnaissance 

•  Battlefield  smoke 

•  Convoy  following,  and 

•  Anti-armor 

The  testbed  is  flexible  and  extensible,  facilitating  the  integration  of  technologies  from 
diverse  sources  such  as: 

•  the  Small  Business  Innovative  Research  program 

•  the  NASA/JPL  Computer  Aided  Remote  Driving  research  effort,  and 

•  DARPA's  Autonomous  Land  Vehicle  and  Image  Understanding  Programs. 

A  more  comprehensive  list  of  research  and  development  topics  that  TACOM  has  been  or  is 
now  involved  in  is: 

•  route  planning 

•  electronic  maps 

•  teleoperation  kits 

•  convoy  controllers 

•  retrotraverse 

•  computer  aided  remote  driving  (CARD) 

•  autonomous  road  following 

•  obstacle  avoidance 

•  intervehicular  cooperation 

•  autonomous  cross  country  navigation. 

Current  and  planned  work  areas,  projects,  and  contracts  and  the  status  of  the  projects  are 
listed  below.  The  largest  project  is  the  control  station  under  development  by  FMC.  This 
has  totaled  $1 1  million  to  date  whereas  the  other  projects  are  $50K  to  $500K.  These 
include  the  following: 


86 


U.S.  Army  Tank-Automotive  Robotics  Research  Work  Units 


Organization 


Smis. 


Objective 

Intelligent  Mobility 

Route  Planner 

Computer  Aided  Remote  Driving 
Autonomous  Mobility 
Outdoor  Scene  Perception 
Robotic  Convoy 
Cooperative  Mission  Planner 
Perception  Planning  Interaction 
UGV  Mobility  Study 
UGV  Platform 
Remote  Control  Kit 
Laser  Navigation 

Soldier  Robot  Interface 
Multiple  &  Mobile  Control 
Station 

Multiple  Vehicle  Control 
Multiple  Sensor  Control  Station 

Sensors 

Stabilized  Platform 
Multiple  Function  Sensor 
Night  Teleoperation  Sensor 
Vision  Laboratory 

Communication 

Reduce  Bandwidth  Microwave 
&  Relieve  Tracking  Antenna 
Electronically  Steerable  Antenna 
Video  Compression 
Video  Compression 
Intelligent  Communication 
Network 

Stacked  Antenna  Array 
Stacked  Antenna  Array 

Mission  Automation 

Forward  Observer  Remote  Target 
Acquisition  System 


KMS  Fusion  Algorithms 
Jet  Propulsion  Laboratory 
Carnegie  Mellon  University 
University  of  Massachusetts 
Redzone  Incorporation 
ADS  Incorporation 
Hughes  Research  Laboratory 
Redzone  Incorporation 
Benthos  Incorporation 
Redzone  Incorporation 
Odetics  Corporation 

FMC  Corporation 

TII  Incorporated 
Fietzsch 


Pietzsch 
Physical  Optics 
Baird 

John  Griffin 


Microwave  Radio  Corporation 
TRA 

Delta  Systems 

TRC 

VIASAT 

Flam  &  Russel 
Atlantic  Aerospace 


Odetics  Corporation 


Completed  &  Transferred  * 
Ongoing  &  Joint  DARPA1 2 
Ongoing  &  DARPA  sponsored^ 
Ongoing  &  DARPA  sponsored^ 
Phase  II  approved 
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FY92  award  &  DARPA  sponsored 
FY92  Phase  I  SBIR 
FY92  Phase  I  SBIR 
FY92  Phase  I  SBIR  Complete 
Algorithms  tested 


Fabrication  Completed  DEMO  I* 

Ongoing  Phase  II  SBIR 
FY92FME 


GFP  to  DARPA  Contracts 
Ongoing  Phase  I  SBIR 
HMMWV  installed  DEMO  1 1 
GMI  Thesis  underway 


Delivered  Installed  VAVCT 
&  Robotic  Lab1 
Delivered  Robotic  Laboratory1 
Phase  II  Complete  FY921 
Phase  II  Complete  FY921 
FY92  Phase  II  SBIR  Award 

Phase  I  Completed  Nov  91 
Phase  I  Completed  Nov  91 


Phase  II  Fabrication  Completed. 
DEMO  I1 


1  in  DEMO  I  support 

2  in  DEMO  II  support 
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Figure  25.  TACOM’s  Contributions  Towards  the  Functionalities  Defined  in 

Figure  7 


7.17  MARTIN  MARIETTA 

Martin  Marietta  has  the  systems  integration  contract  for  Demo  DL  They  plan  to  begin  work 
on  this  $24  million  dollar  program  in  January  1992.  Martin's  previous  major  activity  lies 
in  the  development  and  demonstration  of  the  Autonomous  Land  Vehicle  (ALV)  in  1985-88. 
Since  that  time  they  have  been  funded  largely  by  internal  studies  with  the  Baltimore  group 
doing  some  human  interface  activities  in  cooperation  with  Human  Engineering  Laboratory. 

Martin  Marietta  has  been  instrumental  in  the  development  of  autonomous  navigation 
through  their  development  of  the  Autonomous  Land  Vehicle  (ALVIN)  at  Martin  Marietta 
Denver  Aerospace.  Alvin  is  a  mobile  laboratory  designed  for  demonstrating  the  state-of- 
the-art  in  a  range  of  machine  intelligence  technologies,  including  computer  vision, 
autonomous  navigation,  and  intelligent  planning  and  reasoning. 

The  ALV  mobile  robot  performed  its  First  public  road-following  demonstration  in  May 
1985.  The  basic  perception  sensing  system  was  developed  to  locate  roads  in  video  imagery 
and  send  three  dimensional  road  centerpoints  to  Alvin's  navigation  system.  Since  that  first 
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demonstration,  the  perception  sensing  system  has  matured  into  a  more  general  framework 
for  mobile  robot  vision,  incorporating  both  video  and  range  sensors  and  extending  its  road¬ 
following  capabilities  to  include  obstacle  detection  and  avoidance.  A  second  public 
demonstration  in  June  1986  showed  the  improved  road-following  ability  of  the  system, 
allowing  the  ALV  to  travel  a  distance  of  4.5  km  at  speeds  up  to  10  km/hr,  handle  variations 
in  road  surface  and  navigate  a  sharp,  almost  hairpin,  curve.  In  October  1986,  the  initial 
obstacle  avoidance  capabilities  were  demonstrated,  as  Alvin  maneuvered  around  obstacles 
while  remaining  on  the  road  at  a  speed  of  3.5  km/hr,  and  vehicle  speeds  up  to  20  km/hr 
were  demonstrated  on  an  obstacle-free  portion  of  road. 

Alvin  is  an  all-terrain  vehicle  with  eight-wheel  drive,  diesel-powered,  and  hydrostatically 
driven,  with  a  fiberglass  shell  to  protect  the  interior  from  dust  and  inclement  weather  and  to 
insulate  the  equipment  inside.  The  ALV  hosts  a  number  of  sensors.  A  Land  Navigation 
System  (LNS)  provides  position  and  heading  information.  The  primary  vision  sensor  is  a 
color  video  CCD  camera,  mounted  on  a  pan/tilt  unit  that  is  under  the  direct  control  of  the 
vision  subsystem.  The  other  vision  sensor  is  a  laser  range  scanner  which  determines  range 
by  measuring  the  phase  shift  of  a  reflected  modulated  laser  beam.  The  color  video  camera 
provides  480x512  red,  green,  and  blue  images,  with  eight  bits  of  intensity  per  pixel.  The 
field  of  view  (38°  vertical  and  50°  horizontal)  and  focus  of  the  camera  are  kept  fixed. 

The  laser  range  scanner,  developed  by  the  Environmental  Research  Institute  of  Michigan 
(ERIM),  creates  a  "range  image"  of  the  scene  in  front  of  Alvin.  The  scanner  provides  a 
64x256  array  of  eight  bit  range  values  at  a  rate  of  two  frames  per  second.  The  field  of  view 
is  30°  vertical  by  80°  horizontal.  Range  is  calculated  by  the  scanner  by  measuring  the  phase 
shift  of  a  reflected  modulated  laser  beam,  directed  by  a  pair  of  rotating  mirrors.  The 
resolution  of  range  values  are  good  to  about  three  inches.  Because  of  the  phase  difference 
between  two  signals  to  measure  distance,  the  range  values  obtained  are  relative  and 
inherently  ambiguous  -  objects  whose  distance  to  the  sensor  differ  by  exactly  one 
modulation  cycle  have  the  same  range  value.  For  the  ERIM  sensor,  the  distance 
corresponding  to  one  modulation  cycle  (the  ambiguity  interval)  is  64  feet 

Alvin  currently  uses  a  variety  of  different  computers,  and  the  computer  architecture  has 
been  designed  to  facilitate  the  integration  of  additional  machines  as  necessary.  The  diverse 
processing  requirements  were  met  by  designing  a  modular  multiprocessor  architecture.  In 
the  "first  generation"  ALV  hardware,  VITS  is  hosted  on  a  Vicom  image  processor,  while 
the  other  software  subsystems  are  hosted  on  an  Intel  multiprocessor  system.  VITS 
communicates  with  the  other  subsystems  across  a  dedicated  communication  channel. 

The  vision  subsystem  is  composed  of  three  basic  modules:  VITS,  the  vision  executive, 
which  handles  initialization,  sets  up  communication  channels,  and  "oversees"  the 
processing;  VIVD,  the  video  data  processing  unit;  and  VIRD,  the  range  data  processing 
unit.  The  task  of  the  vision  subsystem  in  road-following  is  to  process  color  or  range 
images  to  produce  a  description  of  the  road  in  front  of  the  vehicle.  This  description  is 
passed  to  die  reasoning  subsystem,  which  uses  additional  data  such  as  current  position, 
speed,  and  heading  to  generate  a  trajectory  for  Alvin  to  follow. 

Communication  between  the  vision  subsystem  and  Reasoning  takes  place  in  two  basic 
forms:  the  scene  model  and  the  position  update.  The  scene  model  is  the  output  of  the 
vision  subsystem  after  each  image  frame  is  processed.  The  scene  model  contains  a  record 
of  Alvin's  position  and  heading  at  the  time  of  image  acquisition,  and  a  description  of  the 
road  found  in  the  image.  The  description  of  the  road  consists  of  lists  of  points  denoting  left 
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and  right  edges  of  the  road,  as  well  as  points  surrounding  obstacles.  The  points  are  given 
as  3D  positions  with  respect  to  the  vehicle  center  of  gravity  at  the  time  the  imagery  was 
acquired.  The  reasoning  subsystem  must  then  transform  the  road  description  into  a  fixed, 
world  coordinate  system  for  navigation. 

VITS  must  know  the  position  and  heading  of  the  vehicle  at  the  time  of  image  acquisition  to 
integrate  sensor  information  acquired  at  different  times,  and  to  transform  vehicle-centered 
data  into  world  coordinates.  In  addition,  VITS  must  be  able  to  predict  the  location  of  the 
road  in  an  image,  given  its  location  in  the  preceding  image.  These  are  effected  by  means  of 
a  position  update  message  passed  from  Reasoning  to  the  vision  subsystem.  The  position 
update  contains  information  on  the  current  vehicle  speed,  position,  and  heading. 
Synchronization  of  position  update  and  image  acquisition  is  mediated  by  a  position  update 
request. 

The  Reasoning  subsystem  is  the  executive  controller  of  the  ALV,  and  Vision  is  a  resource 
of  Reasoning.  At  the  highest  level.  Reasoning  is  responsible  for  receiving  goals  from  a 
human  test  conductor,  creating  a  plan  script  to  accomplish  the  goals,  and  coordinating  the 
other  subsystems  on  Alvin  to  perform  the  necessary  tasks. 

Because  the  processing  involved  in  creating  a  visual  description  of  the  environment  is 
beyond  the  real-time  capability  of  present  computers,  the  scene  model  is  not  used  directly  in 
the  vehicle's  control  servo  loop.  Instead,  the  Navigator  (part  of  the  reasoning  subsystem) 
pieces  together  scene  models  from  the  vision  system  and  builds  a  reference  trajectory  that  is 
sent  to  the  Pilot  for  control.  The  reasoning  subsystem  accepts  a  position  update  request 
from  VITS,  generates  the  appropriate  data,  and  sends  a  position  update  to  VITS.  Upon 
receipt  of  a  scene  model.  Reasoning  evaluates  it  and  plots  a  smooth  trajectory  if  the  data  is 
acceptable.  The  new  trajectory  is  computed  to  smoothly  fit  the  previous  trajectory. 

The  Pilot  performs  the  actual  driving  of  the  vehicle.  Given  a  trajectory  from  Reasoning, 
the  Pilot  computes  the  error  values  of  lateral  position,  heading  and  speed  by  comparing 
LNS  data  with  the  target  values  specified  in  the  trajectory.  The  Pilot  uses  a  table  of 
experimentally  obtained  control  gains  to  determine  commands  needed  to  drive  the  errors 
towards  zero;  these  commands  are  output  to  the  vehicle  controllers.  The  vision  subsystem 
has  no  direct  communication  with  the  Pilot. 

Figure  26  illustrates  Martin- Marietta’s  contributions  towards  development  of  functionalities 
identified  in  Figure  7. 
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Levels  of  Autonomy 

Required  Functionalities  Telepresence  Teleoperation  Teleassislance  Guided  Robotics  Supervised  Autonomous 
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Figure  26.  Martin-Marietta  Corporation’s  Contribution  Towards  the 
Functionalities  Defined  in  Figure  7* 


*  Martin  Marietta  is  and  has  been  a  systems  integrator  for  unmanned  ground  vehicles. 
This  summary  includes  developments  by  subcontractors  which  Martin  Marietta  has 
integrated  into  unmanned  vehicle  systems. 
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